[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 504942139 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2608MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003381] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005010] kvm-guest: setup PV IPIs [ 0.008453] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011131] LSM: Security Framework initializing [ 0.012057] Yama: becoming mindful. [ 0.013031] SELinux: Initializing. [ 0.014064] *** VALIDATE selinux *** [ 0.022705] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026551] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027167] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028094] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029103] *** VALIDATE tmpfs *** [ 0.031003] *** VALIDATE proc *** [ 0.032136] *** VALIDATE cgroup *** [ 0.033008] *** VALIDATE cgroup2 *** [ 0.034345] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035208] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037023] Spectre V2 : User space: Vulnerable [ 0.038007] Speculative Store Bypass: Vulnerable [ 0.041793] debug: unmapping init [mem 0xffffffff89c59000-0xffffffff89c60fff] [ 0.044154] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045686] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046021] ... version: 2 [ 0.047008] ... bit width: 48 [ 0.048006] ... generic registers: 4 [ 0.049007] ... value mask: 0000ffffffffffff [ 0.050008] ... max period: 00007fffffffffff [ 0.051008] ... fixed-purpose events: 3 [ 0.052007] ... event mask: 000000070000000f [ 0.053294] rcu: Hierarchical SRCU implementation. [ 0.055360] smp: Bringing up secondary CPUs ... [ 0.056513] x86: Booting SMP configuration: [ 0.057017] .... node #0, CPUs: #1 #2 #3 [ 0.063273] smp: Brought up 1 node, 4 CPUs [ 0.065018] smpboot: Max logical packages: 1 [ 0.066008] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.137310] node 0 deferred pages initialised in 69ms [ 0.143350] devtmpfs: initialized [ 0.145288] x86/mm: Memory block size: 128MB [ 0.150875] gcov: version magic: 0x41383552 [ 0.154329] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.157113] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.160544] pinctrl core: initialized pinctrl subsystem [ 0.161437] [ 0.161878] ************************************************************* [ 0.165012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.167009] ** ** [ 0.170012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.173009] ** ** [ 0.176012] ** This means that this kernel is built to expose internal ** [ 0.178083] ** IOMMU data structures, which may compromise security on ** [ 0.181012] ** your system. ** [ 0.183019] ** ** [ 0.185009] ** If you see this message and you are not debugging the ** [ 0.187054] ** kernel, report this immediately to your vendor! ** [ 0.190011] ** ** [ 0.192009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.194011] ************************************************************* [ 0.197579] NET: Registered protocol family 16 [ 0.199485] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.203066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.206051] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.209096] cpuidle: using governor menu [ 0.210841] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.213424] PCI: Using configuration type 1 for base access [ 0.215118] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.224111] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.227060] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.230156] cryptd: max_cpu_qlen set to 1000 [ 0.233265] ACPI: Added _OSI(Module Device) [ 0.234000] ACPI: Added _OSI(Processor Device) [ 0.234000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.236012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.241253] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.247409] ACPI: Interpreter enabled [ 0.249051] ACPI: PM: (supports S0 S3 S4 S5) [ 0.250020] ACPI: Using IOAPIC for interrupt routing [ 0.252114] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.255499] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.266452] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.269037] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.271015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.275075] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.280466] acpiphp: Slot [2] registered [ 0.282119] acpiphp: Slot [5] registered [ 0.283095] acpiphp: Slot [6] registered [ 0.285146] acpiphp: Slot [7] registered [ 0.286106] acpiphp: Slot [8] registered [ 0.288104] acpiphp: Slot [9] registered [ 0.289090] acpiphp: Slot [10] registered [ 0.291075] acpiphp: Slot [3] registered [ 0.292085] acpiphp: Slot [4] registered [ 0.294074] acpiphp: Slot [11] registered [ 0.295094] acpiphp: Slot [12] registered [ 0.297081] acpiphp: Slot [13] registered [ 0.298085] acpiphp: Slot [14] registered [ 0.300070] acpiphp: Slot [15] registered [ 0.301082] acpiphp: Slot [16] registered [ 0.302077] acpiphp: Slot [17] registered [ 0.304097] acpiphp: Slot [18] registered [ 0.305139] acpiphp: Slot [19] registered [ 0.307100] acpiphp: Slot [20] registered [ 0.308136] acpiphp: Slot [21] registered [ 0.309079] acpiphp: Slot [22] registered [ 0.311091] acpiphp: Slot [23] registered [ 0.312076] acpiphp: Slot [24] registered [ 0.314105] acpiphp: Slot [25] registered [ 0.315082] acpiphp: Slot [26] registered [ 0.317070] acpiphp: Slot [27] registered [ 0.318064] acpiphp: Slot [28] registered [ 0.320090] acpiphp: Slot [29] registered [ 0.321137] acpiphp: Slot [30] registered [ 0.324171] acpiphp: Slot [31] registered [ 0.326146] PCI host bridge to bus 0000:00 [ 0.328025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.331063] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.333018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.337042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.341062] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.345025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.348363] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.352114] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.355506] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.364750] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.369606] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.372020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.374037] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.376013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.379351] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.383602] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.387050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.391793] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.396012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.411019] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.416013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.422359] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.433014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.446014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.464013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.477193] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.485014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.492011] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.512012] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.524000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.532016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.539014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.544014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.555830] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.564014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.571016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.591014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.605000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.615013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.621014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.637016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.648000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.659014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.669242] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.695017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.716509] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.718929] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.721391] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.723394] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.725179] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.731181] iommu: Default domain type: Passthrough [ 0.733433] SCSI subsystem initialized [ 0.734173] ACPI: bus type USB registered [ 0.736089] usbcore: registered new interface driver usbfs [ 0.738065] usbcore: registered new interface driver hub [ 0.740085] usbcore: registered new device driver usb [ 0.742220] pps_core: LinuxPPS API ver. 1 registered [ 0.744010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.746082] PTP clock support registered [ 0.749124] EDAC MC: Ver: 3.0.0 [ 0.750164] PCI: Using ACPI for IRQ routing [ 0.751000] NetLabel: Initializing [ 0.753010] NetLabel: domain hash size = 128 [ 0.754007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.756130] NetLabel: unlabeled traffic allowed by default [ 0.759232] vgaarb: loaded [ 0.760555] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.762010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.768308] clocksource: Switched to clocksource kvm-clock [ 0.883825] VFS: Disk quotas dquot_6.6.0 [ 0.885359] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.887851] *** VALIDATE ramfs *** [ 0.888817] *** VALIDATE hugetlbfs *** [ 0.890521] pnp: PnP ACPI init [ 0.905237] pnp: PnP ACPI: found 6 devices [ 0.929248] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.932752] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.935012] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.937325] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.939919] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.942346] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.945216] NET: Registered protocol family 2 [ 0.947686] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.952754] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.956503] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.962071] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.965083] TCP: Hash tables configured (established 65536 bind 65536) [ 0.967686] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.970653] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.973425] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.976448] NET: Registered protocol family 1 [ 0.979138] RPC: Registered named UNIX socket transport module. [ 0.981536] RPC: Registered udp transport module. [ 0.982801] RPC: Registered tcp transport module. [ 0.984361] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.987175] NET: Registered protocol family 44 [ 0.988521] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.990373] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.992400] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.994906] PCI: CLS 0 bytes, default 64 [ 0.996677] Unpacking initramfs... [ 2.449518] debug: unmapping init [mem 0xffff88fcbcc54000-0xffff88fcbffbffff] [ 2.453210] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.454796] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.457462] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.006929] Initialise system trusted keyrings [ 3.009032] Key type blacklist registered [ 3.011072] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.021515] zbud: loaded [ 3.026715] *** VALIDATE nfs *** [ 3.029452] *** VALIDATE nfs4 *** [ 3.032320] pstore: using deflate compression [ 3.038436] Platform Keyring initialized [ 3.168797] NET: Registered protocol family 38 [ 3.170872] Key type asymmetric registered [ 3.172477] Asymmetric key parser 'x509' registered [ 3.174870] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.178480] io scheduler mq-deadline registered [ 3.180362] io scheduler kyber registered [ 3.182414] io scheduler bfq registered [ 3.184396] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.187783] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.190597] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.194143] ACPI: Power Button [PWRF] [ 3.203369] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.216720] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.239698] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.250377] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.281425] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.311568] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.341785] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.347506] Non-volatile memory driver v1.3 [ 3.349323] Linux agpgart interface v0.103 [ 3.384562] virtio_blk virtio1: [vda] 134768 512-byte logical blocks (69.0 MB/65.8 MiB) [ 3.387619] vda: detected capacity change from 0 to 69001216 [ 3.450237] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.468809] vdb: detected capacity change from 0 to 1073741824 [ 3.494369] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.499082] vdc: detected capacity change from 0 to 2621440000 [ 3.514341] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.516654] vdd: detected capacity change from 0 to 2621440000 [ 3.532151] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.537463] vde: detected capacity change from 0 to 4294967296 [ 3.560610] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.565664] vdf: detected capacity change from 0 to 4294967296 [ 3.573927] libphy: Fixed MDIO Bus: probed [ 3.583603] usbcore: registered new interface driver usbserial_generic [ 3.585574] usbserial: USB Serial support registered for generic [ 3.587875] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.591685] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.593405] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.596264] mousedev: PS/2 mouse device common for all mice [ 3.603168] rtc_cmos 00:05: RTC can wake from S4 [ 3.607353] rtc_cmos 00:05: registered as rtc0 [ 3.607365] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.612162] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.616336] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.622549] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.623674] intel_pstate: CPU model not supported [ 3.630810] hid: raw HID events driver (C) Jiri Kosina [ 3.633846] usbcore: registered new interface driver usbhid [ 3.636350] usbhid: USB HID core driver [ 3.638627] drop_monitor: Initializing network drop monitor service [ 3.641951] Initializing XFRM netlink socket [ 3.644766] NET: Registered protocol family 10 [ 3.648353] Segment Routing with IPv6 [ 3.649947] NET: Registered protocol family 17 [ 3.652392] mpls_gso: MPLS GSO support [ 3.663612] RAS: Correctable Errors collector initialized. [ 3.666976] AVX version of gcm_enc/dec engaged. [ 3.669351] AES CTR mode by8 optimization enabled [ 3.785197] sched_clock: Marking stable (3785147859, 0)->(4722204138, -937056279) [ 3.790679] registered taskstats version 1 [ 3.793909] Loading compiled-in X.509 certificates [ 3.798695] zswap: loaded using pool lzo/zbud [ 3.840255] Key type big_key registered [ 3.852527] Key type encrypted registered [ 3.854440] ima: No TPM chip found, activating TPM-bypass! [ 3.857064] ima: Allocated hash algorithm: sha1 [ 3.859161] ima: No architecture policies found [ 3.860981] evm: Initialising EVM extended attributes: [ 3.862839] evm: security.selinux [ 3.863735] evm: security.ima [ 3.864589] evm: security.capability [ 3.865757] evm: HMAC attrs: 0x1 [ 3.868471] rtc_cmos 00:05: setting system clock to 2026-05-25 05:34:32 UTC (1779687272) [ 3.875899] debug: unmapping init [mem 0xffffffff8ac03000-0xffffffff8adfffff] [ 3.879358] debug: unmapping init [mem 0xffffffff89982000-0xffffffff89c58fff] [ 3.888415] Write protecting the kernel read-only data: 28672k [ 3.892458] debug: unmapping init [mem 0xffffffff88003000-0xffffffff881fffff] [ 3.895552] debug: unmapping init [mem 0xffffffff88914000-0xffffffff889fffff] [ 3.934475] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.942142] systemd[1]: Detected virtualization kvm. [ 3.943444] systemd[1]: Detected architecture x86-64. [ 3.944733] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.980987] systemd[1]: No hostname configured. [ 3.983265] systemd[1]: Set hostname to . [ 3.985602] random: systemd: uninitialized urandom read (16 bytes read) [ 3.988513] systemd[1]: Initializing machine ID from random generator. [ 4.044423] random: ln: uninitialized urandom read (6 bytes read) [ 4.135505] random: systemd: uninitialized urandom read (16 bytes read) [ 4.137616] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.145245] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.152432] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Sockets. [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.152405] device-mapper: uevent: version 1.0.3 [ 5.155533] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ 6.891169] random: fast init done [ 6.928905] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Started udev Coldplug all Devices. [ 7.589340] scsi host0: ata_piix [ 7.678143] scsi host1: ata_piix [ 7.679929] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.693176] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 Starting dracut initqueue hook... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 13.374208] random: crng init done [ 13.375576] random: 7 urandom warning(s) missed due to ratelimiting [ 14.624570] dracut-initqueue[569]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 16.317336] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 18.093509] printk: systemd: 26 output lines suppressed due to ratelimiting [ 18.440526] SELinux: Disabled at runtime. [ 18.514106] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.523296] systemd[1]: Detected virtualization kvm. [ 18.525445] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 19.373211] systemd[1]: initrd-switch-root.service: Succeeded. [ 19.377327] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 19.382845] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 19.386460] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 19.389473] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 19.403396] systemd[1]: Starting Journal Service... Starting Journal Service... [ 19.410624] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [[ 19.609192] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS  OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 20.618540] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.430618] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.471227] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 22.089889] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.245514] EDAC sbridge: Ver: 1.1.2 [ 25.216692] Key type dns_resolver registered [ 25.562229] NFS: Registering the id_resolver key type [ 25.567224] Key type id_resolver registered [ 25.569151] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (6s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon...[ 30.662012] hrtimer: interrupt took 13101608 ns Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg313-server login: [ 101.569415] libcfs: loading out-of-tree module taints kernel. [ 101.614852] Key type ._llcrypt registered [ 101.617671] Key type .llcrypt registered [ 101.715456] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 118.669420] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 120.639490] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 120.656536] alg: No test for adler32 (adler32-zlib) [ 122.146116] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 122.950633] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 124.727225] Key type lgssc registered [ 126.570615] Lustre: Echo OBD driver; http://www.lustre.org/ [ 142.205378] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 177.287609] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 191.609797] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 191.639139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.883637] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 192.918143] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.041360] Lustre: lustre-MDT0000: new disk, initializing [ 193.154907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.189113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 197.075649] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.493360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 208.613449] Lustre: 6493:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 208.657758] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 208.663180] Lustre: Skipped 1 previous similar message [ 208.788983] Lustre: lustre-MDT0001: new disk, initializing [ 208.862608] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 208.904380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 208.920064] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 212.832460] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 217.739780] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 227.195401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 227.458828] Lustre: lustre-OST0000: new disk, initializing [ 227.463211] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 227.475674] Lustre: 8396:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 227.542992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 233.282032] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.589475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 235.599443] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 235.646804] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 246.110130] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 246.246751] Lustre: lustre-OST0001: new disk, initializing [ 246.249436] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 246.253139] Lustre: 9450:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 246.312538] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 252.143354] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.076849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 256.091771] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 256.158157] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 262.646704] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 270.321732] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 275.599919] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing check_logdir /tmp/testlogs/ [ 280.115463] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing yml_node [ 284.392708] Lustre: DEBUG MARKER: Client: 2.17.53.24 [ 286.556776] Lustre: DEBUG MARKER: MDS: 2.17.53.24 [ 288.688501] Lustre: DEBUG MARKER: OSS: 2.17.53.24 [ 289.975958] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon May 25 01:39:17 EDT 2026 [ 308.792714] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 310.434630] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 328.160317] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.162426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.183877] Lustre: Skipped 1 previous similar message [ 328.203667] Lustre: Skipped 3 previous similar messages [ 333.213256] Lustre: server umount lustre-MDT0000 complete [ 336.652620] LustreError: 6485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779687605 with bad export cookie 3684462086447552292 [ 336.672938] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 336.680308] LustreError: 6485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 338.400822] LustreError: 6499:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 338.400843] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.401634] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 338.420290] LustreError: 6499:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 338.429636] Lustre: Skipped 3 previous similar messages [ 341.407929] LustreError: 6501:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 341.407947] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 341.440320] Lustre: Skipped 2 previous similar messages [ 343.052435] Lustre: server umount lustre-MDT0001 complete [ 347.229233] Lustre: server umount lustre-OST0000 complete [ 350.797985] Lustre: server umount lustre-OST0001 complete [ 357.283742] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 366.552331] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 399.755594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 399.978519] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 400.000365] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 400.066773] Lustre: lustre-MDT0000: new disk, initializing [ 400.187278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 400.219052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 404.487576] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 414.260931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 414.318641] Lustre: 14963:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 414.359191] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 414.365924] Lustre: Skipped 1 previous similar message [ 414.444960] Lustre: lustre-MDT0001: new disk, initializing [ 414.553447] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 414.603725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 414.631242] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 419.366368] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 427.486792] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 432.341213] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 438.076330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 438.406241] Lustre: lustre-OST0000: new disk, initializing [ 438.409377] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 438.413713] Lustre: 16707:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 438.502337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.689941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 439.708085] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 439.776133] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 445.908716] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 453.145435] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 457.684684] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 457.988978] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 461.883213] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 462.060940] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 465.377914] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 465.379913] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 465.403128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 465.418492] Lustre: Skipped 1 previous similar message [ 470.503876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 470.512934] Lustre: Skipped 1 previous similar message [ 478.175431] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 478.335768] Lustre: server umount lustre-OST0000 complete [ 485.857735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.875095] Lustre: Skipped 1 previous similar message [ 485.882198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 485.886073] Lustre: Skipped 3 previous similar messages [ 489.074765] Lustre: server umount lustre-MDT0000 complete [ 490.976112] LustreError: 16206:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 490.996642] LustreError: 16206:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 493.597851] LustreError: 15744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779687762 with bad export cookie 3684462086447553825 [ 493.605310] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 493.613400] LustreError: 15744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 494.150412] Lustre: server umount lustre-MDT0001 complete [ 500.930357] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:42:47 (1779687767) [ 507.756726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 508.456695] LustreError: 18825:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 508.580580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 512.778040] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 514.016553] LustreError: 18826:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 520.645609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 524.760660] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 529.683172] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 533.153711] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 539.604879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 545.331860] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 549.870755] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 558.207727] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 564.707772] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 564.722972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 564.729640] Lustre: Skipped 3 previous similar messages [ 564.751228] Lustre: Skipped 2 previous similar messages [ 570.356338] Lustre: server umount lustre-MDT0000 complete [ 573.579429] LustreError: 18806:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779687842 with bad export cookie 3684462086447554700 [ 573.580827] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 573.595454] LustreError: 18806:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 574.060854] Lustre: server umount lustre-MDT0001 complete [ 588.090591] Lustre: server umount lustre-OST0000 complete [ 607.074082] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 616.234333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 616.692199] LustreError: 22852:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 616.707635] LustreError: 22852:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 616.781592] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 616.790149] Lustre: Skipped 2 previous similar messages [ 621.551372] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 629.766235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 634.440082] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 637.512868] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 643.790365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 649.488897] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 657.130610] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.269814] Lustre: lustre-OST0001: new disk, initializing [ 657.274275] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 657.282594] Lustre: 25119:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 659.376395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 659.388182] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 659.477286] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 662.793608] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 671.637384] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 674.620953] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 686.560788] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 686.567193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 686.577907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 686.588055] Lustre: Skipped 3 previous similar messages [ 691.586419] Lustre: server umount lustre-MDT0000 complete [ 692.196724] LustreError: 22847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 692.220905] LustreError: 22847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 694.807990] LustreError: 22833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779687963 with bad export cookie 3684462086447555554 [ 694.809923] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 694.818024] LustreError: 22833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 695.195547] Lustre: server umount lustre-MDT0001 complete [ 708.943837] Lustre: server umount lustre-OST0000 complete [ 721.520784] Lustre: server umount lustre-OST0001 complete [ 736.495151] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 744.580419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 744.917194] LustreError: 28103:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 744.990442] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 744.996638] Lustre: Skipped 3 previous similar messages [ 748.583550] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 755.983647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 760.070990] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 762.557721] Lustre: 29219:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 768.263884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 774.834216] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 782.806515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 788.944352] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 795.924560] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 799.019990] Lustre: 31062:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 807.904068] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 807.918127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 807.941786] Lustre: Skipped 3 previous similar messages [ 807.959050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 807.965756] Lustre: Skipped 4 previous similar messages [ 810.351601] Lustre: server umount lustre-MDT0000 complete [ 813.747460] LustreError: 28084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779688082 with bad export cookie 3684462086447556842 [ 813.759562] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 813.773950] LustreError: 28084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 814.049195] LustreError: 30274:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 814.059351] LustreError: 30274:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 814.370908] Lustre: server umount lustre-MDT0001 complete [ 823.962242] Lustre: server umount lustre-OST0000 complete [ 827.295429] Lustre: server umount lustre-OST0001 complete [ 835.083789] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:48:21 (1779688101) [ 861.756670] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 872.549440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 877.944834] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 886.884502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 891.148873] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 893.786432] Lustre: 34958:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 899.342335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 904.349849] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 911.721554] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 918.120670] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 925.586591] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 928.641568] Lustre: 36803:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 933.857586] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 933.884108] Lustre: Skipped 7 previous similar messages [ 938.469129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 938.486734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 938.496936] Lustre: Skipped 7 previous similar messages [ 939.745366] Lustre: server umount lustre-MDT0000 complete [ 943.161649] LustreError: 33824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779688211 with bad export cookie 3684462086447557955 [ 943.169071] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 943.194397] LustreError: 33824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 943.554436] Lustre: server umount lustre-MDT0001 complete [ 958.173573] Lustre: server umount lustre-OST0000 complete [ 959.462549] Lustre: 3648:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779688212/real 1779688212] req@ffff88fc0250b100 x1866137486030720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779688228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 961.474850] Lustre: server umount lustre-OST0001 complete [ 969.702722] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:50:36 (1779688236) [ 984.465657] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 993.685375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 994.135190] LustreError: 38931:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 994.150940] LustreError: 38931:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 997.937485] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1004.287117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1004.694165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1004.698520] Lustre: Skipped 8 previous similar messages [ 1008.200444] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1010.661532] Lustre: 40043:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1016.431353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1022.989196] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1030.846407] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1036.500449] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1042.765544] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1045.713295] Lustre: 41887:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1046.468953] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1046.471854] Lustre: Skipped 1 previous similar message [ 1050.592210] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1050.604143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1050.617072] Lustre: Skipped 2 previous similar messages [ 1055.123879] Lustre: server umount lustre-MDT0000 complete [ 1058.301882] LustreError: 38913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779688326 with bad export cookie 3684462086447559061 [ 1058.311354] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1058.314754] LustreError: 38913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1064.679341] Lustre: server umount lustre-MDT0001 complete [ 1071.428544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1087.630422] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1092.970791] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1094.849520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1098.942720] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1100.269302] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1100.276283] Lustre: Skipped 1 previous similar message [ 1103.760255] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1105.104669] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1108.031778] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1109.257690] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1115.625893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1119.256856] Lustre: server umount lustre-MDT0000 complete [ 1122.273056] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779688390 with bad export cookie 3684462086447560188 [ 1122.274687] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1122.285802] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1122.585948] Lustre: server umount lustre-MDT0001 complete [ 1135.756979] Lustre: server umount lustre-OST0000 complete [ 1148.620698] Lustre: server umount lustre-OST0001 complete [ 1157.667901] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:53:44 (1779688424) [ 1172.818498] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1182.783637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.170968] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1195.165463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1199.829801] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1202.542310] Lustre: 47366:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1209.288600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1214.987910] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1223.337288] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1229.293408] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1235.901309] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1239.486317] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1248.248449] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1248.268845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1248.286025] Lustre: Skipped 12 previous similar messages [ 1248.300178] Lustre: Skipped 22 previous similar messages [ 1250.493563] Lustre: server umount lustre-MDT0000 complete [ 1253.345951] LustreError: 46249:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1253.381845] LustreError: 46249:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 37 previous similar messages [ 1253.867623] LustreError: 47368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779688522 with bad export cookie 3684462086447560972 [ 1253.875965] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1253.884925] LustreError: 47368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1254.314497] Lustre: server umount lustre-MDT0001 complete [ 1268.088511] Lustre: server umount lustre-OST0000 complete [ 1283.056658] Lustre: server umount lustre-OST0001 complete [ 1291.397875] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:55:58 (1779688558) [ 1293.737826] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1295.571453] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:56:02 (1779688562) [ 1303.426557] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 1309.925971] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1327.806396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1328.096787] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1328.182760] Lustre: lustre-MDT0000: new disk, initializing [ 1328.323044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1331.876505] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1339.155800] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1346.114193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1346.491770] Lustre: 53261:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1347.903219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1347.930304] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1352.405600] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1358.990029] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1361.135741] Lustre: DEBUG MARKER: create test files [ 1389.933397] Lustre: server umount lustre-OST0000 complete [ 1393.337191] Lustre: server umount lustre-MDT0000 complete [ 1402.163087] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 1404.440188] Key type lgssc unregistered [ 1404.663822] LNet: 54760:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1404.670673] LNetError: 54760:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1404.687493] LNet: Removed LNI 192.168.203.113@tcp [ 1405.336140] Key type .llcrypt unregistered [ 1405.337798] Key type ._llcrypt unregistered [ 1427.544156] Key type ._llcrypt registered [ 1427.548689] Key type .llcrypt registered [ 1427.675933] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1428.625975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1428.654519] alg: No test for adler32 (adler32-zlib) [ 1429.641533] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 1429.819338] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 1431.487180] Key type lgssc registered [ 1432.409981] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1438.295660] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1438.322626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1439.676916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1442.697234] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1447.044709] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1452.975456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1453.278612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1458.535334] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1462.900512] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1464.554838] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1464.631340] Lustre: 57343:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1464.638971] Lustre: 57343:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1464.644547] Lustre: 57343:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1464.649199] Lustre: 57343:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1464.653437] Lustre: 57343:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1464.657328] Lustre: 57343:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1472.458913] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1476.668629] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1485.282135] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1485.283259] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1485.312264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1487.429443] Lustre: server umount lustre-OST0000 complete [ 1490.647990] Lustre: server umount lustre-MDT0000 complete [ 1499.296450] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 1502.059171] Key type lgssc unregistered [ 1502.384906] LNet: 58819:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1502.392398] LNetError: 58819:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1502.413252] LNet: Removed LNI 192.168.203.113@tcp [ 1503.174851] Key type .llcrypt unregistered [ 1503.178897] Key type ._llcrypt unregistered [ 1527.841235] Key type ._llcrypt registered [ 1527.844860] Key type .llcrypt registered [ 1527.937826] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1528.912423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1528.930362] alg: No test for adler32 (adler32-zlib) [ 1529.912467] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 1530.152076] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 1531.807884] Key type lgssc registered [ 1532.815216] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1538.813875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1538.832313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1540.144927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1543.229634] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1547.711054] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1553.453257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1553.692745] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1558.950332] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1563.763840] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1564.971797] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1565.079096] Lustre: 61501:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1565.086680] Lustre: 61501:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1565.092251] Lustre: 61501:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1565.097738] Lustre: 61501:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1565.101766] Lustre: 61501:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1565.108723] Lustre: 61501:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1573.949732] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1585.636344] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1585.643527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1585.657093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1588.721079] Lustre: server umount lustre-OST0000 complete [ 1592.090988] Lustre: server umount lustre-MDT0000 complete [ 1601.487869] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 1603.723890] Key type lgssc unregistered [ 1603.998837] LNet: 62941:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1604.005354] LNetError: 62941:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1604.022247] LNet: Removed LNI 192.168.203.113@tcp [ 1604.725430] Key type .llcrypt unregistered [ 1604.728531] Key type ._llcrypt unregistered [ 1618.664048] Key type ._llcrypt registered [ 1618.668450] Key type .llcrypt registered [ 1618.774921] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 1633.474602] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1634.080639] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1634.111845] alg: No test for adler32 (adler32-zlib) [ 1635.112055] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 1635.287962] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 1636.935367] Key type lgssc registered [ 1637.832713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1670.302517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1670.324230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1671.574315] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1671.599193] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1671.663707] Lustre: lustre-MDT0000: new disk, initializing [ 1671.771642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1671.801692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1675.571407] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1683.791844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1683.879800] Lustre: 66062:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1683.916746] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1683.920440] Lustre: Skipped 1 previous similar message [ 1683.984313] Lustre: lustre-MDT0001: new disk, initializing [ 1684.098346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1684.125753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1684.149700] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1687.497080] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1695.774687] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1698.967715] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1706.503036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1706.707802] Lustre: lustre-OST0000: new disk, initializing [ 1706.711797] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1706.718474] Lustre: 67805:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1706.804839] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1712.039469] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1712.198666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1712.223692] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1712.319125] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1718.016424] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1721.651207] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1721.856350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1725.344463] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1725.568829] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1727.969830] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1727.980041] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1727.985425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1727.994685] Lustre: Skipped 1 previous similar message [ 1728.012092] Lustre: Skipped 1 previous similar message [ 1733.097130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1738.210359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1738.218449] Lustre: Skipped 2 previous similar messages [ 1741.279513] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1741.386292] Lustre: server umount lustre-OST0000 complete [ 1745.380383] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1745.386784] LustreError: Skipped 1 previous similar message [ 1745.389284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1745.397684] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1748.450226] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1750.708322] Lustre: server umount lustre-MDT0000 complete [ 1754.302715] LustreError: 66842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779689022 with bad export cookie 17184138300668949355 [ 1754.310070] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1754.312554] LustreError: 66842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1754.629360] Lustre: server umount lustre-MDT0001 complete [ 1763.253857] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:03:49 (1779689029) [ 1772.784859] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1808.779669] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 1818.756154] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 1847.615268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1847.797368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1847.817119] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1847.887103] Lustre: lustre-MDT0000: new disk, initializing [ 1847.946029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.958474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1851.513536] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1861.124111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1861.260514] Lustre: 73624:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1861.307764] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1861.311662] Lustre: Skipped 1 previous similar message [ 1861.395806] Lustre: lustre-MDT0001: new disk, initializing [ 1861.501665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1861.530803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1861.541343] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1865.453145] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1871.861639] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1875.336121] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1882.013683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1882.359229] Lustre: lustre-OST0000: new disk, initializing [ 1882.363798] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1882.379268] Lustre: 75366:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1882.473092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1887.360675] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1891.458786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1891.479641] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1891.610034] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1895.178428] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1898.289123] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1898.481440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1901.867742] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1901.974613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1907.170093] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1907.174764] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1907.176645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1907.176652] Lustre: Skipped 2 previous similar messages [ 1907.186274] Lustre: Skipped 1 previous similar message [ 1909.432977] Lustre: server umount lustre-OST0000 complete [ 1912.803860] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1912.809838] LustreError: Skipped 1 previous similar message [ 1912.812110] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1917.408837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1917.416911] Lustre: Skipped 3 previous similar messages [ 1918.619796] LustreError: 76757:0:(ldlm_resource.c:1180:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x5:0x0].0x0 (0000000057860f9f) refcount nonzero (0) after lock cleanup; forcing cleanup. [ 1918.721133] Lustre: server umount lustre-MDT0000 complete [ 1921.804244] LustreError: 73660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779689190 with bad export cookie 17184138300668950230 [ 1921.808133] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1921.813151] LustreError: 73660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1922.440421] Lustre: server umount lustre-MDT0001 complete [ 1930.218962] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:06:37 (1779689197) [ 1935.929034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1936.378505] LustreError: 77630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1936.403195] LustreError: 77630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1936.486467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1940.470760] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1941.476410] LustreError: 77631:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1946.610559] LustreError: 77630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1949.204245] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1953.735848] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1958.191790] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1961.387440] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1967.269817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1972.841886] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 1978.351696] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1982.270801] Lustre: 79961:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 5c2b640e-11d5-4c01-943e-33a7896223ab at adminstrative request [ 1987.041657] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1987.051877] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1997.292433] Lustre: MGS: Client 5c2b640e-11d5-4c01-943e-33a7896223ab (at 0@lo) reconnecting [ 1997.328335] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xee7a52bd2e8eba48 to 0xee7a52bd2e8ebcef [ 1997.354078] Lustre: MGC192.168.203.113@tcp: Connection restored to 0@lo (at 0@lo) [ 2019.985964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2020.237609] Lustre: lustre-OST0001: new disk, initializing [ 2020.252719] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2020.260078] Lustre: 80355:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2020.375610] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2020.381385] Lustre: Skipped 2 previous similar messages [ 2026.458268] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2028.234849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2028.246195] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2028.394859] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2033.217449] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2038.757119] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2038.763200] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2038.767929] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2038.791290] Lustre: Skipped 2 previous similar messages [ 2038.803293] Lustre: Skipped 1 previous similar message [ 2050.015196] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2050.090569] Lustre: server umount lustre-OST0001 complete [ 2053.600474] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2053.611133] LustreError: Skipped 1 previous similar message [ 2054.121196] LustreError: 81300:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2054.138723] LustreError: 81300:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2059.247926] LustreError: 80803:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2059.270437] LustreError: 80803:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2059.552419] Lustre: server umount lustre-OST0000 complete [ 2064.352827] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2064.361983] Lustre: Skipped 3 previous similar messages [ 2069.192176] Lustre: server umount lustre-MDT0000 complete [ 2069.472443] LustreError: 78889:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2069.483837] LustreError: 78889:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2072.065749] LustreError: 77610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779689340 with bad export cookie 17184138300668951791 [ 2072.071537] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2072.080535] LustreError: 77610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2072.306218] Lustre: server umount lustre-MDT0001 complete [ 2081.321110] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:09:07 (1779689347) [ 2083.176435] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2085.247924] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:09:11 (1779689351) [ 2087.832812] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2089.937630] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:09:16 (1779689356) [ 2091.798716] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2093.878272] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:09:20 (1779689360) [ 2095.595856] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2097.309489] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:09:23 (1779689363) [ 2098.913739] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2104.973454] loop: module loaded [ 2104.980652] loop0: detected capacity change from 0 to 4294967296 [ 2105.043966] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2106.547697] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2109.742577] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2113.108732] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2136.166405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2136.213633] LustreError: 84348:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2136.219696] LustreError: 84348:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2136.224818] LustreError: 84348:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2136.233633] LustreError: 84348:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2153.717565] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 2163.300677] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 2196.803772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2197.090976] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2197.142534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2197.259960] Lustre: lustre-MDT0000: new disk, initializing [ 2197.386044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2197.422059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2201.600840] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2211.530554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2211.588276] Lustre: 87178:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2211.630220] LustreError: 87190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2211.822338] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2215.328148] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2222.411939] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2227.239608] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2233.979784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2234.145647] Lustre: 88923:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2240.353440] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2243.724966] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2243.837412] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2247.358362] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2250.956743] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2251.137959] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2255.090108] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2255.361784] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2259.427959] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2259.444530] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2259.460224] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2259.462915] Lustre: Skipped 11 previous similar messages [ 2271.199155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2271.409634] Lustre: server umount lustre-OST0000 complete [ 2280.834537] Lustre: server umount lustre-MDT0000 complete [ 2284.152983] LustreError: 87170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779689552 with bad export cookie 17184138300668952407 [ 2284.164689] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2284.170300] LustreError: 87170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2284.526350] Lustre: server umount lustre-MDT0001 complete [ 2291.910862] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:12:38 (1779689558) [ 2297.613328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2298.035318] LustreError: 91190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2302.146588] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2309.472038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2313.285497] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2318.270622] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2322.204862] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2328.718094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2329.139408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2329.150446] Lustre: Skipped 4 previous similar messages [ 2334.863588] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2339.727943] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2343.625677] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2343.830826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2348.055708] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2348.247363] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2353.721476] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2353.850160] Lustre: lustre-OST0001: new disk, initializing [ 2353.852240] Lustre: Skipped 2 previous similar messages [ 2353.854748] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2353.857498] Lustre: Skipped 3 previous similar messages [ 2353.861286] Lustre: 94185:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2359.915970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2359.930212] Lustre: Skipped 2 previous similar messages [ 2359.939760] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2360.034657] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2360.124823] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2366.958801] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2370.878382] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2371.087243] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2374.704733] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2374.904726] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2391.372669] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2411.534306] Lustre: Failing over lustre-MDT0000 [ 2412.000642] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2412.011018] LustreError: Skipped 1 previous similar message [ 2412.014319] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2412.021180] Lustre: Skipped 3 previous similar messages [ 2412.024426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2412.026948] Lustre: Skipped 9 previous similar messages [ 2413.983751] Lustre: server umount lustre-MDT0000 complete [ 2416.610461] LustreError: 91185:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2416.640307] LustreError: 91185:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 2432.981561] Lustre: 64284:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779689685/real 1779689685] req@000000000ce5d62d x1866139071406208/t0(0) o400->MGC192.168.203.113@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1779689701 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2433.012555] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2435.457758] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2435.462204] LDISKFS-fs (dm-0): recovery complete [ 2435.486590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2443.238505] LustreError: 64283:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000065a576b7 x1866139071415680/t0(0) o250->MGC192.168.203.113@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2443.600745] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2446.982308] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2447.434251] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2447.440242] Lustre: 92802:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2448.874906] Lustre: 92219:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2448.884469] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2448.885548] Lustre: 92219:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 2448.957167] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2449.279217] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2449.617946] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2449.626097] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2449.951101] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2449.960772] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2449.971617] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2449.977481] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 2450.931935] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2450.946659] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 3 previous similar messages [ 2451.271959] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2451.281764] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 3 previous similar messages [ 2452.254068] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2452.262968] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 6 previous similar messages [ 2453.216628] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2453.228171] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2453.551183] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2453.564212] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2456.415944] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2456.422511] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2457.388380] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2457.407199] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2457.735805] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2457.751854] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2464.696757] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2464.713834] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 25 previous similar messages [ 2465.670680] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2465.675608] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2465.991614] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2465.999578] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2480.851667] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2480.867937] Lustre: 96882:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 50 previous similar messages [ 2481.816847] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2481.831534] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2482.159327] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2482.173772] LustreError: 96882:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2507.039167] Lustre: 96882:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2507.051241] Lustre: 96882:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2507.061544] Lustre: 96882:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ebeb1501-bdbe-4014-ad8e-fc5f9eb93c31@192.168.203.13@tcp [ 2507.075081] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2507.085276] Lustre: 96882:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@000000004a5e75d5 x1866139056543360/t0(8589934780) o35->ebeb1501-bdbe-4014-ad8e-fc5f9eb93c31@192.168.203.13@tcp:11/0 lens 392/0 e 0 to 0 dl 1779689786 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2507.113325] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2507.115077] Lustre: lustre-MDT0000: Denying connection for new client ebeb1501-bdbe-4014-ad8e-fc5f9eb93c31 (at 192.168.203.13@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2507.134991] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2507.169174] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2507.175603] Lustre: Skipped 2 previous similar messages [ 2507.177769] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2507.206382] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2507.212414] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2517.987422] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2522.605932] Lustre: server umount lustre-OST0000 complete [ 2531.785581] Lustre: server umount lustre-OST0001 complete [ 2540.710052] Lustre: server umount lustre-MDT0000 complete [ 2543.348425] LustreError: 91172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779689811 with bad export cookie 17184138300669003626 [ 2543.351554] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2543.362060] LustreError: 91172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2543.667889] Lustre: server umount lustre-MDT0001 complete [ 2549.817096] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:16:56 (1779689816) [ 2555.273216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2555.277983] Lustre: *** cfs_fail_loc=197, val=0*** [ 2570.783839] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2575.903532] LustreError: 98838:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.113@tcp: failed processing log, type 4: rc = -110 [ 2581.041921] Lustre: *** cfs_fail_loc=197, val=0*** [ 2581.044879] Lustre: Skipped 65 previous similar messages [ 2601.567233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2601.571592] Lustre: Skipped 2 previous similar messages [ 2605.846248] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2610.085486] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2612.031480] Lustre: server umount lustre-OST0000 complete [ 2618.721557] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:18:05 (1779689885) [ 2636.613660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2636.869578] LustreError: 100344:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2636.880632] LustreError: 100344:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2639.768343] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2645.129318] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2648.172563] LustreError: 100375:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2648.184808] LustreError: 100375:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2648.195832] LustreError: 100375:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -5 [ 2648.495481] Lustre: server umount lustre-MDT0000 complete [ 2672.152939] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 2680.615490] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 2708.838339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2709.075904] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2709.116701] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2709.218337] Lustre: lustre-MDT0000: new disk, initializing [ 2709.343600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2712.970242] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2721.957267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2722.125960] Lustre: 104398:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2722.428657] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2726.096605] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2731.711951] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2734.206641] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2738.759327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2738.944833] Lustre: 106139:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2743.012163] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2745.479993] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2748.440161] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2751.284798] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2751.429194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2754.741621] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2754.914570] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2760.676040] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2760.679720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2760.689298] LustreError: Skipped 1 previous similar message [ 2760.697398] Lustre: Skipped 9 previous similar messages [ 2760.701726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2760.706792] Lustre: Skipped 13 previous similar messages [ 2762.491964] Lustre: server umount lustre-OST0000 complete [ 2770.906746] Lustre: server umount lustre-MDT0000 complete [ 2773.312340] LustreError: 104391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690041 with bad export cookie 17184138300669014322 [ 2773.315127] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2773.320938] LustreError: 104391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2773.547459] Lustre: server umount lustre-MDT0001 complete [ 2779.413573] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:20:46 (1779690046) [ 2785.043749] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 2786.888352] Key type lgssc unregistered [ 2787.123840] LNet: 108459:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2787.134822] LNetError: 108459:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2787.153739] LNet: Removed LNI 192.168.203.113@tcp [ 2787.720791] Key type .llcrypt unregistered [ 2787.723060] Key type ._llcrypt unregistered [ 2801.556238] Key type ._llcrypt registered [ 2801.558658] Key type .llcrypt registered [ 2801.643105] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 2815.366485] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 2816.512852] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2816.545594] alg: No test for adler32 (adler32-zlib) [ 2817.590543] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 2817.825220] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 2819.487322] Key type lgssc registered [ 2820.362509] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2856.061890] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2856.087137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2857.325384] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2857.388601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2857.504741] Lustre: lustre-MDT0000: new disk, initializing [ 2857.610958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2857.638712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2861.397491] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2870.148193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2870.354929] Lustre: lustre-OST0000: new disk, initializing [ 2870.358051] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2870.361188] Lustre: Skipped 1 previous similar message [ 2870.366152] Lustre: 112581:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2870.424313] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2871.872568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2871.880375] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2871.918394] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2875.240458] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2884.165534] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2884.323591] Lustre: lustre-OST0001: new disk, initializing [ 2884.327770] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2884.332323] Lustre: 113428:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2884.419582] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2889.905138] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 2891.395963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2891.405188] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2891.432372] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2901.474398] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2901.484400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2901.487632] Lustre: Skipped 1 previous similar message [ 2903.745331] Lustre: server umount lustre-MDT0000 complete [ 2909.453848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2913.269427] LustreError: 111785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690181 with bad export cookie 18087624197195119301 [ 2913.280404] LustreError: 111785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2913.281296] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2913.350329] Lustre: server umount lustre-OST0000 complete [ 2916.866770] Lustre: server umount lustre-OST0001 complete [ 2925.008024] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 02:23:11 (1779690191) [ 2933.666359] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 2951.579750] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:23:38 (1779690218) [ 2982.123894] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 2990.180720] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3016.671952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3016.894192] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3016.923734] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3016.975211] Lustre: lustre-MDT0000: new disk, initializing [ 3017.084059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3017.095421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3020.324047] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3029.353815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3029.491350] Lustre: 120099:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3029.560836] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3029.565177] Lustre: Skipped 1 previous similar message [ 3029.609940] Lustre: lustre-MDT0001: new disk, initializing [ 3029.704991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3029.751420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3029.766177] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3033.396540] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3039.601903] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3042.915949] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3048.333963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3048.540708] Lustre: lustre-OST0000: new disk, initializing [ 3048.546453] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3048.554859] Lustre: 121842:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3048.635804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3053.303471] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3057.246091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3057.254821] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3057.350466] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3059.321698] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3062.931156] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3063.221270] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3067.140409] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3067.350711] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3072.991963] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3072.996588] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3073.003875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3073.016353] Lustre: Skipped 1 previous similar message [ 3073.021533] Lustre: Skipped 1 previous similar message [ 3078.124115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3078.140054] Lustre: Skipped 2 previous similar messages [ 3083.232770] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3083.240516] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3083.261621] Lustre: Skipped 1 previous similar message [ 3083.475430] Lustre: server umount lustre-OST0000 complete [ 3088.355355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3088.365556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3088.375241] Lustre: Skipped 1 previous similar message [ 3088.392304] Lustre: Skipped 1 previous similar message [ 3092.650155] Lustre: server umount lustre-MDT0000 complete [ 3095.839502] LustreError: 120091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690364 with bad export cookie 18087624197195120519 [ 3095.848746] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3095.851166] LustreError: 120091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3096.238968] Lustre: server umount lustre-MDT0001 complete [ 3103.309198] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:26:10 (1779690370) [ 3108.742047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3109.124935] LustreError: 124106:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3109.145672] LustreError: 124106:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3109.207279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3112.543268] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3114.466695] LustreError: 124107:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3118.562950] LustreError: 124106:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3119.080512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3122.633399] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3127.217467] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3130.199300] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3135.094125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3139.378536] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3143.409165] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3157.301149] Lustre: *** cfs_fail_loc=159, val=0*** [ 3173.511526] Lustre: lustre-MDT0000: Client f4b04ddf-69d5-4ea4-abb8-6fb9192ebe65 (at 192.168.203.13@tcp) reconnecting [ 3174.587660] Lustre: *** cfs_fail_loc=159, val=0*** [ 3174.592023] Lustre: Skipped 5 previous similar messages [ 3190.894447] Lustre: lustre-MDT0000: Client f4b04ddf-69d5-4ea4-abb8-6fb9192ebe65 (at 192.168.203.13@tcp) reconnecting [ 3210.215114] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3210.215520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3210.222499] LustreError: Skipped 1 previous similar message [ 3210.233663] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3212.882677] Lustre: server umount lustre-OST0000 complete [ 3216.868929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3216.876224] LustreError: Skipped 1 previous similar message [ 3216.879287] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3216.889287] Lustre: Skipped 1 previous similar message [ 3221.925195] Lustre: server umount lustre-MDT0000 complete [ 3224.647971] LustreError: 124088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690493 with bad export cookie 18087624197195121408 [ 3224.651184] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3224.665545] LustreError: 124088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3224.936286] Lustre: server umount lustre-MDT0001 complete [ 3233.603823] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3235.740318] Key type lgssc unregistered [ 3236.005617] LNet: 127545:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3236.016706] LNetError: 127545:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3236.037887] LNet: Removed LNI 192.168.203.113@tcp [ 3236.787965] Key type .llcrypt unregistered [ 3236.790528] Key type ._llcrypt unregistered [ 3253.434410] Key type ._llcrypt registered [ 3253.436238] Key type .llcrypt registered [ 3253.503926] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3254.234538] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3254.310673] alg: No test for adler32 (adler32-zlib) [ 3255.309889] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3255.498396] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3257.143825] Key type lgssc registered [ 3257.969892] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3263.392508] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3263.403600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3264.749184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3267.914161] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3274.372578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3274.673820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3277.818833] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3282.016492] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3284.718833] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3289.181564] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3289.392786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3290.415177] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3293.147543] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3296.978803] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3307.551288] Lustre: *** cfs_fail_loc=159, val=0*** [ 3322.988820] Lustre: lustre-MDT0000: Client bb5b13cb-4066-48ce-ae4b-75f80d6398c9 (at 192.168.203.13@tcp) reconnecting [ 3324.487804] Lustre: *** cfs_fail_loc=159, val=0*** [ 3340.908559] Lustre: lustre-MDT0001: Client bb5b13cb-4066-48ce-ae4b-75f80d6398c9 (at 192.168.203.13@tcp) reconnecting [ 3341.948068] Lustre: *** cfs_fail_loc=159, val=0*** [ 3341.957341] Lustre: Skipped 3 previous similar messages [ 3358.320576] Lustre: lustre-MDT0001: Client bb5b13cb-4066-48ce-ae4b-75f80d6398c9 (at 192.168.203.13@tcp) reconnecting [ 3373.434939] Lustre: 131639:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3375.501442] Lustre: *** cfs_fail_loc=159, val=0*** [ 3375.506352] Lustre: Skipped 4 previous similar messages [ 3391.073253] Lustre: lustre-MDT0000: Client 3a4d1b96-8f45-4716-b1c9-12ccdab2f571 (at 192.168.203.13@tcp) reconnecting [ 3392.004475] Lustre: *** cfs_fail_loc=159, val=0*** [ 3392.007360] Lustre: Skipped 14 previous similar messages [ 3408.493258] Lustre: lustre-MDT0000: Client 3a4d1b96-8f45-4716-b1c9-12ccdab2f571 (at 192.168.203.13@tcp) reconnecting [ 3409.395306] Lustre: 131931:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3427.812851] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3427.814730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3427.820664] Lustre: Skipped 1 previous similar message [ 3427.838153] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3432.689831] Lustre: server umount lustre-OST0000 complete [ 3438.048460] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3438.053987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3438.065431] Lustre: Skipped 1 previous similar message [ 3438.066672] Lustre: Skipped 1 previous similar message [ 3443.170018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3443.175397] Lustre: Skipped 2 previous similar messages [ 3448.289270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3448.295788] Lustre: Skipped 1 previous similar message [ 3449.312798] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3449.408309] Lustre: server umount lustre-MDT0000 complete [ 3451.488325] LustreError: 131760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690720 with bad export cookie 255461817293538189 [ 3451.488576] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3451.497196] LustreError: 131760:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3451.644687] Lustre: server umount lustre-MDT0001 complete [ 3457.385969] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3459.181898] Key type lgssc unregistered [ 3459.388957] LNet: 132991:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3459.394483] LNetError: 132991:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3459.404035] LNet: Removed LNI 192.168.203.113@tcp [ 3459.875847] Key type .llcrypt unregistered [ 3459.878443] Key type ._llcrypt unregistered [ 3472.864616] Key type ._llcrypt registered [ 3472.866137] Key type .llcrypt registered [ 3472.921717] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3473.407221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3473.482365] alg: No test for adler32 (adler32-zlib) [ 3474.360194] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3474.483569] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3476.095220] Key type lgssc registered [ 3476.649794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3480.138189] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3480.150918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3481.415285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3483.295187] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3487.279498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3487.451943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3489.340913] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3492.016651] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3493.859569] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3497.523275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3497.719865] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3500.641781] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3503.082599] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3503.560595] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3507.174630] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3511.257288] Lustre: 136605:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3512.437856] Lustre: 136653:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3513.070522] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3515.171617] Lustre: 136800:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3515.649220] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3515.653278] Lustre: Skipped 1 previous similar message [ 3516.115788] Lustre: 136898:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3518.435307] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3518.445366] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3518.453657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3520.184119] Lustre: server umount lustre-OST0000 complete [ 3522.528457] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3522.530534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3522.539689] Lustre: Skipped 1 previous similar message [ 3522.548272] Lustre: Skipped 1 previous similar message [ 3527.649063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3527.656156] Lustre: Skipped 1 previous similar message [ 3531.801565] Lustre: server umount lustre-MDT0000 complete [ 3532.770099] LustreError: 134361:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3532.780884] LustreError: 134361:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3533.945334] LustreError: 135083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690802 with bad export cookie 4261096849286172638 [ 3533.946112] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3533.954346] LustreError: 135083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3534.127670] Lustre: server umount lustre-MDT0001 complete [ 3539.594440] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3541.107285] Key type lgssc unregistered [ 3541.305577] LNet: 137939:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3541.312322] LNetError: 137939:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3541.328117] LNet: Removed LNI 192.168.203.113@tcp [ 3541.808604] Key type .llcrypt unregistered [ 3541.810722] Key type ._llcrypt unregistered [ 3554.216474] Key type ._llcrypt registered [ 3554.218126] Key type .llcrypt registered [ 3554.270231] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3554.778741] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3554.799825] alg: No test for adler32 (adler32-zlib) [ 3555.743513] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3555.873217] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3557.495168] Key type lgssc registered [ 3558.200820] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3562.597344] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3562.615950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3563.915108] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3566.160189] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3570.387219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3570.606677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3572.381975] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3575.155853] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3577.075496] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3580.410175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3580.626751] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3581.674439] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3583.671483] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3586.830495] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3588.845618] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3594.823545] Lustre: *** cfs_fail_loc=159, val=0*** [ 3594.829334] Lustre: Skipped 1 previous similar message [ 3610.214758] Lustre: lustre-MDT0000: Client db281dda-d3c7-4718-9904-cb1a2e818c8d (at 192.168.203.13@tcp) reconnecting [ 3612.640430] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3612.644542] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3612.654547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3614.693096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3614.696794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3614.699855] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3614.715431] Lustre: Skipped 1 previous similar message [ 3618.422992] Lustre: server umount lustre-OST0000 complete [ 3621.856504] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3621.864482] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3621.879718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3624.929141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3624.930735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3626.821905] Lustre: server umount lustre-MDT0000 complete [ 3629.119784] LustreError: 140038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779690897 with bad export cookie 16219495398462903965 [ 3629.124310] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3629.127849] LustreError: 140038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3629.327081] Lustre: server umount lustre-MDT0001 complete [ 3635.297344] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3636.905457] Key type lgssc unregistered [ 3637.086742] LNet: 142619:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3637.093079] LNetError: 142619:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3637.108728] LNet: Removed LNI 192.168.203.113@tcp [ 3637.628674] Key type .llcrypt unregistered [ 3637.632883] Key type ._llcrypt unregistered [ 3651.289357] Key type ._llcrypt registered [ 3651.291991] Key type .llcrypt registered [ 3651.355759] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3651.762771] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3651.828766] alg: No test for adler32 (adler32-zlib) [ 3652.743309] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3652.857519] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3654.463170] Key type lgssc registered [ 3655.056208] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3659.757065] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3659.768072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3661.082272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3663.492020] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3668.362334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3668.540566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3670.281080] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3673.217566] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3675.089943] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3678.608700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3678.695557] LustreError: 145595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3678.705492] LustreError: 145595:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3678.875880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3681.847079] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3684.328755] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3684.660235] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3686.388889] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3705.091536] Lustre: lustre-OST0000: evicting e76474ff-5ee6-4e1c-967e-78a5cc1eacfc (at 192.168.203.13@tcp) by administrative request [ 3705.095906] Lustre: Skipped 1 previous similar message [ 3732.449069] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3732.452197] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3732.455966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3732.465522] Lustre: Skipped 1 previous similar message [ 3732.474785] Lustre: Skipped 1 previous similar message [ 3736.904372] Lustre: server umount lustre-OST0000 complete [ 3740.128543] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3740.131184] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3740.136760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3742.688623] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3742.690208] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3742.707703] Lustre: Skipped 1 previous similar message [ 3745.134503] Lustre: server umount lustre-MDT0000 complete [ 3747.808530] LustreError: 144707:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3747.825862] LustreError: 144707:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3749.297672] LustreError: 143975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779691017 with bad export cookie 9566480488582905584 [ 3749.300700] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3749.313951] LustreError: 143975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3749.807240] Lustre: server umount lustre-MDT0001 complete [ 3761.968367] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3765.105567] Key type lgssc unregistered [ 3765.432649] LNet: 147727:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3765.442094] LNetError: 147727:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3765.459399] LNet: Removed LNI 192.168.203.113@tcp [ 3766.737253] Key type .llcrypt unregistered [ 3766.740339] Key type ._llcrypt unregistered [ 3829.490081] Key type ._llcrypt registered [ 3829.491921] Key type .llcrypt registered [ 3829.566461] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3830.193466] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3830.204854] alg: No test for adler32 (adler32-zlib) [ 3831.141647] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3831.307889] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3832.943158] Key type lgssc registered [ 3833.570168] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3839.145894] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3839.165746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3840.572829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3843.477733] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3848.836194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3849.101088] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3851.722359] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3855.424648] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3857.663409] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3862.241618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3862.578127] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3865.651268] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3866.656630] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3870.388803] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3873.837734] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3880.929814] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3880.938159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3880.961644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3884.002893] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3884.004162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3884.018233] Lustre: Skipped 1 previous similar message [ 3886.800820] Lustre: server umount lustre-OST0000 complete [ 3890.144339] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3890.149366] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3890.163631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3894.248118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3894.248442] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3896.046902] Lustre: server umount lustre-MDT0000 complete [ 3898.541312] LustreError: 152338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779691167 with bad export cookie 14354615023986076266 [ 3898.547371] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3898.557227] LustreError: 152338:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3898.748113] Lustre: server umount lustre-MDT0001 complete [ 3905.462443] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 3907.495595] Key type lgssc unregistered [ 3907.693521] LNet: 153359:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3907.699425] LNetError: 153359:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3907.711360] LNet: Removed LNI 192.168.203.113@tcp [ 3908.333634] Key type .llcrypt unregistered [ 3908.337906] Key type ._llcrypt unregistered [ 3924.200320] Key type ._llcrypt registered [ 3924.202983] Key type .llcrypt registered [ 3924.283599] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3925.040534] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3925.058599] alg: No test for adler32 (adler32-zlib) [ 3926.041223] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 3926.229369] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 3927.863255] Key type lgssc registered [ 3928.756882] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3940.254858] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:40:07 (1779691207) [ 3950.455594] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 3956.653737] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 3981.664551] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3981.720534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3982.921723] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3982.959269] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3983.018386] Lustre: lustre-MDT0000: new disk, initializing [ 3983.076690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3983.091460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3986.453312] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 3994.745455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3994.852928] Lustre: 157545:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3994.916483] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3994.922984] Lustre: Skipped 1 previous similar message [ 3994.994209] Lustre: lustre-MDT0001: new disk, initializing [ 3995.075175] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3995.112433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3995.130377] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3998.505522] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4004.582532] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4007.264300] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4012.070685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4012.267221] Lustre: lustre-OST0000: new disk, initializing [ 4012.270990] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4012.278167] Lustre: 159287:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4012.359348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4016.918549] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4019.225811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4019.235613] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4019.309892] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4022.839832] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4032.076331] Lustre: server umount lustre-OST0000 complete [ 4036.067571] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4036.072566] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4036.083205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4039.648581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4039.649382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4039.665232] Lustre: Skipped 1 previous similar message [ 4044.773191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4044.782401] Lustre: Skipped 1 previous similar message [ 4049.375157] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4049.500983] Lustre: server umount lustre-MDT0000 complete [ 4050.400122] Lustre: 154046:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779691303/real 1779691303] req@000000005ffbe9d4 x1866141473471744/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1779691319 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4050.422865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4051.789160] LustreError: 157537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779691320 with bad export cookie 9137304723026778266 [ 4051.795304] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4051.796909] LustreError: 157537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4051.966831] Lustre: server umount lustre-MDT0001 complete [ 4057.657823] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 4059.187319] Key type lgssc unregistered [ 4059.373587] LNet: 161063:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4059.377574] LNetError: 161063:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4059.393293] LNet: Removed LNI 192.168.203.113@tcp [ 4059.853365] Key type .llcrypt unregistered [ 4059.855143] Key type ._llcrypt unregistered [ 4073.861446] Key type ._llcrypt registered [ 4073.864574] Key type .llcrypt registered [ 4073.957557] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 4074.621514] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4074.638618] alg: No test for adler32 (adler32-zlib) [ 4075.656366] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 4075.887199] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 4077.519148] Key type lgssc registered [ 4078.170939] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4082.613889] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4082.641423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4083.911851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4085.946641] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4090.405594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4090.607841] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4092.781444] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4095.591651] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4097.381711] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4101.015208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4101.223692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4104.305182] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4107.367336] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4108.399047] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4122.079533] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4122.093041] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4122.107763] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4122.113474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.123850] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4122.712717] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4122.728441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4122.735783] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4122.741426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.749288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4123.261627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.693150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.146642] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4124.150428] Lustre: Skipped 2 previous similar messages [ 4124.159106] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4124.163391] Lustre: Skipped 2 previous similar messages [ 4124.165453] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4124.167774] Lustre: Skipped 2 previous similar messages [ 4124.172771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.179528] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4124.182014] Lustre: Skipped 2 previous similar messages [ 4124.749705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.237306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.722309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.218796] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4126.224205] Lustre: Skipped 3 previous similar messages [ 4126.234601] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4126.240619] Lustre: Skipped 3 previous similar messages [ 4126.244127] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4126.248834] Lustre: Skipped 3 previous similar messages [ 4126.255562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.261503] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4126.269214] Lustre: Skipped 3 previous similar messages [ 4126.783242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.286729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.806494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.277633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.823842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.294917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.738637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.267888] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4130.275347] Lustre: Skipped 7 previous similar messages [ 4130.291518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4130.302545] Lustre: Skipped 7 previous similar messages [ 4130.307157] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4130.313709] Lustre: Skipped 7 previous similar messages [ 4130.316590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.326673] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4130.333230] Lustre: Skipped 7 previous similar messages [ 4130.919327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.412836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.876746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.371568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.885161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.345476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.897142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.428713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.953891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.511678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.039052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.447465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.911575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.354472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.868383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.316834] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4138.324157] Lustre: Skipped 15 previous similar messages [ 4138.339350] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4138.344344] Lustre: Skipped 15 previous similar messages [ 4138.346352] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4138.349079] Lustre: Skipped 15 previous similar messages [ 4138.351304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.359673] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4138.367386] Lustre: Skipped 15 previous similar messages [ 4138.912699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.449952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.918930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.397561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.885270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.345855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.776968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.239430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.727754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.213484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.682356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.116599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.645098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.119534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.575437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.165744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.687377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.195680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.652287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.134071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.593324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.050941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.453043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.851518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.251058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.701880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.160691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.607599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.066618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.526965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.015574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.498961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.987775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.520664] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4154.523834] Lustre: Skipped 33 previous similar messages [ 4154.533597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4154.545051] Lustre: Skipped 33 previous similar messages [ 4154.550515] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4154.557346] Lustre: Skipped 33 previous similar messages [ 4154.560075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.568184] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4154.573732] Lustre: Skipped 33 previous similar messages [ 4155.042448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.592084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.126871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.653971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.130395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.622022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.213910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.722271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.205401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.691850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.193826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.707509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.269523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.818152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.421830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.914820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.424393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.985367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.543441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.030896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.529413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.052737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.574995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.079816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.583056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.083302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.586617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.081783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.645229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.144531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.705249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.286084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.863216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.447785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.053697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.589569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.142020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.657436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.181318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.718555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.229495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.768047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.306170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.810291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.304621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.829691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.333586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.875745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.413568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.898289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.419204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.901144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.378837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.906822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.418033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.873481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.421492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.969115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.479258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.998069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.528808] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4186.533256] Lustre: Skipped 60 previous similar messages [ 4186.542944] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4186.550844] Lustre: Skipped 60 previous similar messages [ 4186.554505] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4186.557730] Lustre: Skipped 60 previous similar messages [ 4186.560177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.569550] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4186.573855] Lustre: Skipped 60 previous similar messages [ 4187.084882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.560184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.118743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.603956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.135345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.635494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.128924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.623586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.146788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.706239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.233514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.775934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.295597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.802144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.286495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.848107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.352128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.845631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.365766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.912199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.413432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.901255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.398871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.936215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.452174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.037863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.560457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.036284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.525847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.992934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.509585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.023140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.554830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.071738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.561541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.062545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.540765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.036690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.558919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.175899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.774246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.380290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.884718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.469180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.052466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.667722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.299715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.865476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.372208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.924815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.520155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.081164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.619829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.178810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.722283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.266298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.802936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.442885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.070175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.674216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.198271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.711196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.261161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.680794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.148923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.638572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.174051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.710409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.245809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.733650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.227453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.733404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.228901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.742466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.228358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.745798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.220162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.717850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.204536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.682890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.125763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.629464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.109141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.589567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.121743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.610199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.073076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.517394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.983638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.447640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.920528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.435794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.935211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.471865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.960046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.450487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.938980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.451228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.949426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.433172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.882571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.351783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.861954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.385803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.906566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.434129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.940796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.479183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.001409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.561762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.103817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.641112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.219689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.700159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.172622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.715360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.207932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.720639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.360953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.909714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.485936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.027319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.564758] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4250.569514] Lustre: Skipped 122 previous similar messages [ 4250.578886] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4250.588993] Lustre: Skipped 122 previous similar messages [ 4250.593390] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4250.597328] Lustre: Skipped 122 previous similar messages [ 4250.602317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.608591] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4250.614134] Lustre: Skipped 122 previous similar messages [ 4251.097911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.658580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.213792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.847625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.476142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.075517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.607297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.136665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.631969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.132131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.638436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.100637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.586404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.062966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.546532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.023112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.555253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.026199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.565686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.110318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.611730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.057741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.602789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.068988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.518351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.059227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.523574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.987342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.487888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.968421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.426218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.926891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.412684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.860524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.306742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.790318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.210688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.592389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.028646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.486101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.944401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.390149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.846880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.287120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.685631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.177240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.654796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.089300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.494327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.996828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.466926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.924065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.376195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.797897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.244987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.721890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.180800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.654381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.194578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.710135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.149255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.609104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.060205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.500895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.951271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.429900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.886034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.315889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.750522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.234406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.736799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.253466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.764129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.259032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.691497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.198592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.708169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.253091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.754732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.309859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.817916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.321104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.759250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.169910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.635646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.089519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.555113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.031065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.546864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.049933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.550773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.020928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.454660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.967313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.427106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.861395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.280076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.748153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.166358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.621843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.074040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.490408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.907539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.357316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.830410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.293820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.744984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.172372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.638786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.119488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.596743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.056933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.538440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.989371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.453213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.936611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.415791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.853755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.294473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.724530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.132846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.564300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.950771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.401830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.850775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.242396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.632111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.032622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.448398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.873044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.347935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.846150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.324386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.819196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.302579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.753199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.261032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.729297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.205616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.701308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.188373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.681080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.156890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.631442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.081350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.526746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.006922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.516976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.017467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.506617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.959776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.432409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.905580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.381216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.900441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.320249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.783049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.261981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.757989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.256539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.728577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.165405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.570285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.001851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.443214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.888678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.329501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.717429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.181981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.609232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.005350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.382951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.740942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.123193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.506266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.893773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.292412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.778040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.201818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.590410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.968058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.385930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.777767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.207406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.637370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.020774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.428056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.847625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.251439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.716621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.153870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.514634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.887467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.287171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.655744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.021094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.376721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.741946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.087455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.462670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.818851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.171865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.551415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.920850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.303251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.714454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.075557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.498155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.897112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.299679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.722786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.131874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.557768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.992435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.430040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.893752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.332757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.815174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.215441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.626593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.015713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.402073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.791335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.197372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.592843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.994140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.467437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.884718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.305558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.757491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.177354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.651214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.076334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.486929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.912246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.338654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.713609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.125946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.527133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.002597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.443186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.884508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.278556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.753800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.205134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.612446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.017313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.432111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.830433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.236137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.635254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.012791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.395209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.776745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.210468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.675437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.136364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.542619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.996391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.461372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.862210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.284311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.716750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.168749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.539787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.953437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.311703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.672306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.059054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.451829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.857069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.300454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.667587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.024874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.400770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.810770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.231748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.690092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.101850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.513721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.919742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.326032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.737574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.159123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.512884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.853833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.192850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.556625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.976413] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4378.981053] Lustre: Skipped 288 previous similar messages [ 4378.990412] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4378.996910] Lustre: Skipped 288 previous similar messages [ 4378.999725] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4379.002889] Lustre: Skipped 288 previous similar messages [ 4379.005334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.010786] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4379.013914] Lustre: Skipped 288 previous similar messages [ 4379.492504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.906196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.319182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.707343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.100557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.513578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.891870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.258607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.612514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.979971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.365226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.746148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.107908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.500902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.909072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.313216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.699117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.041175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.468248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.880244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.306882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.691545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.031299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.400836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.735308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.102474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.457857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.831251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.208545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.592407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.961164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.343326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.707666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.076210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.455433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.819444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.187062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.542251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.925032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.282885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.641680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.016248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.355039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.699532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.044406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.383351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.707239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.044082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.400082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.765499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.115058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.442981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.784370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.130602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.475460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.846609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.240443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.609109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.982204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.376636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.754287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.115281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.476066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.845519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.213622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.568606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.956896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.320310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.687864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.071326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.451094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.827541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.211372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.579181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.922799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.273019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.610419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.967143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.344839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.719365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.104960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.501713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.916996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.289301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.676927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.028854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.425146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.817687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.254073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.680119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.120821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.570327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.989906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.382842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.737517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.135721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.537145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.913733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.298389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.671881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.042191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.399688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.777960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.136381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.485499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.835568] LustreError: 162459:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4418.840170] LustreError: 162459:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4418.840876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.213423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.579103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.975448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.368604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.749635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.153341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.551661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.952597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.387060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.810292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.203252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.611746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.024918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.434913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.821943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.211704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.610193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.028347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.473827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.889788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.321069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.739107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.153823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.547254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.909349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.254367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.574411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.916801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.247179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.606129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.985228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.350080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.738386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.108036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.482174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.832255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.183817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.529843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.912223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.290694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.656222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.040889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.416503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.796551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.168100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.520470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.867747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.205373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.573590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.899190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.256046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.618428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.957186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.319667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.741078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.127334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.521640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.928666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.303729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.668871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.024532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.368908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.722876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.099426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.490257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.892921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.299505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.762059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.178425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.628625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.131118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.564548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.961536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.356267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.756613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.134799] LustreError: 162459:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4448.139244] LustreError: 162459:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4448.140231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.528073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.905508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.264810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.639258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.044702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.448045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.888426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.298641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.747071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.160158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.560557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.010555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.455491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.919245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.364780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.834046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.210227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.647352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.084432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.447473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.864452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.279705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.684322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.146028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.636402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.095979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.514196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.936557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.334189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.763752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.172705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.600322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.029654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.428460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.821043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.228292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.632834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.006131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.368684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.723319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.099511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.465558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.852995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.238499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.671674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.070604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.469955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.900683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.328708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.728158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.166505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.583395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.021368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.456074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.863463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.270833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.746515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.229728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.677581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.156738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.565245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.015945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.437046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.890524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.320684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.729836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.130963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.511740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.918400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.309665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.705655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.127492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.593897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.046853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.444544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.811155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.185234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.559416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.946397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.350440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.741922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.166559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.541365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.918916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.304495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.686452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.141764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.538444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.901634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.288662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.690944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.089971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.483076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.872081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.215211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.538884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.877515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.229237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.598919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.975778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.387592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.775191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.107298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.443155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.824383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.187110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.562185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.953458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.331843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.713612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.105423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.544304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.933604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.362268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.747407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.107735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.527370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.915749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.356060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.775359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.183207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.564907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.966920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.354635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.790620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.186629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.626373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.993789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.392604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.772857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.222689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.653314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.074573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.487361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.991285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.401470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.762323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.172093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.608241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.987481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.389100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.752618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.132154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.492750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.874756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.279774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.657072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.027128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.417622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.775621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.159839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.579421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.056360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.451147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.819124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.193216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.574381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.971354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.350709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.723378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.124324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.535211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.929786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.352492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.810763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.232071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.612298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.028540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.409457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.795383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.235551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.645419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.039947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.426097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.822238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.194991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.590191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.026023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.448305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.850549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.220285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.603678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.016931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.404514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.742476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.094719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.440522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.780383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.127196] LustreError: 162459:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4524.132650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.133664] LustreError: 162459:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4524.483510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.841154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.193203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.552778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.897683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.234474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.578360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.941800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.371151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.789756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.173200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.550665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.949855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.340433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.694793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.057938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.403093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.760838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.149577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.573269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.026764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.416993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.819478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.200096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.623360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.012123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.394187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.783258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.192193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.590153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.975361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.379346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.773216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.173976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.582305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.018869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.405726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.791337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.188870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.608985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.999314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.405144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.829631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.190497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.542039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.941632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.284333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.633826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.001628] LustreError: 162459:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4543.006621] LustreError: 162459:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4543.007524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.438451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.846664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.237816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.625682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.004869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.408213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.883057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.275433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.638511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.051518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.463854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.900140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.321059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.744647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.138245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.561665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.957894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.377708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.835789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.237431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.680153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.063955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.470581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.924506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.307224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.697200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.098703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.514568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.888434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.299413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.734474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.130980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.540102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.936648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.326155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.723701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.109464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.454699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.808441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.194263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.131733] LustreError: 164031:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4779.135527] LustreError: 162459:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4901.866497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4901.866898] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4901.866989] Lustre: Skipped 460 previous similar messages [ 4901.868928] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4901.870417] LustreError: Skipped 4 previous similar messages [ 4904.419567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4904.420671] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4904.426101] LustreError: Skipped 2 previous similar messages [ 4906.139694] Lustre: server umount lustre-OST0000 complete [ 4909.537737] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4909.541407] Lustre: Skipped 2 previous similar messages [ 4914.656136] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4914.661284] Lustre: Skipped 1 previous similar message [ 4919.776281] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4919.779882] Lustre: Skipped 1 previous similar message [ 4922.335101] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4922.462924] Lustre: server umount lustre-MDT0000 complete [ 4924.062375] LustreError: 162422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692192 with bad export cookie 15188639643716907512 [ 4924.063686] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4924.072764] LustreError: 162422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4924.217238] Lustre: server umount lustre-MDT0001 complete [ 4928.876497] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 4930.797728] Key type lgssc unregistered [ 4931.076328] LNet: 217167:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4931.084660] LNetError: 217167:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4932.146855] LNet: Removed LNI 192.168.203.113@tcp [ 4932.782319] Key type .llcrypt unregistered [ 4932.784257] Key type ._llcrypt unregistered [ 4953.687243] Key type ._llcrypt registered [ 4953.688968] Key type .llcrypt registered [ 4953.747385] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 4954.276551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4954.365087] alg: No test for adler32 (adler32-zlib) [ 4955.452916] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 4955.715661] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 4957.375232] Key type lgssc registered [ 4958.261439] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4964.358469] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4964.378258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4965.839869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4969.862559] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4976.101729] LustreError: 218564:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4976.115654] LustreError: 218564:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4977.124741] LustreError: 218565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4978.574451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4978.842537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4982.988782] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 4987.952410] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4993.223544] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4999.538515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4999.864774] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5005.318833] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5009.967844] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5012.174852] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5019.323922] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5020.970628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.987516] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5021.000529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.022480] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5021.023108] LustreError: 218584:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5022.702090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5022.707038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5022.720395] Lustre: Skipped 1 previous similar message [ 5026.921759] Lustre: lustre-OST0000: Not available for connect from 192.168.203.13@tcp (stopping) [ 5026.925734] Lustre: Skipped 1 previous similar message [ 5028.893950] Lustre: server umount lustre-OST0000 complete [ 5041.063383] LustreError: 218584:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5060.078966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5060.324698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5063.989393] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5067.190929] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5069.205411] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 5071.572129] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ce543218000.ost_server_uuid 50 [ 5072.480443] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ce543218000.ost_server_uuid in FULL state after 0 sec [ 5077.858107] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5077.863299] Lustre: Skipped 3 previous similar messages [ 5080.965096] Lustre: server umount lustre-OST0000 complete [ 5086.696657] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5086.708240] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5086.726547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5089.612472] Lustre: server umount lustre-MDT0000 complete [ 5092.492544] LustreError: 218546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692361 with bad export cookie 7065180663257125620 [ 5092.496111] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5092.500775] LustreError: 218546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5092.796925] Lustre: server umount lustre-MDT0001 complete [ 5100.269265] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 5102.351362] Key type lgssc unregistered [ 5102.559103] LNet: 223207:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5102.570378] LNetError: 223207:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5102.585464] LNet: Removed LNI 192.168.203.113@tcp [ 5103.133527] Key type .llcrypt unregistered [ 5103.135672] Key type ._llcrypt unregistered [ 5122.145484] Key type ._llcrypt registered [ 5122.148168] Key type .llcrypt registered [ 5122.224977] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5122.910067] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5122.956972] alg: No test for adler32 (adler32-zlib) [ 5123.915910] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 5124.095223] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 5125.703227] Key type lgssc registered [ 5126.343876] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5132.102788] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5132.112933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5133.426158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5135.745469] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5141.472068] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5141.711174] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5144.692913] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5147.140116] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5151.980120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5152.173045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5156.205570] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5159.359350] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5162.473040] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5162.609304] Lustre: lustre-OST0001: new disk, initializing [ 5162.613298] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5162.620142] Lustre: 227156:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5162.682893] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5167.437683] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5168.717107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5168.725420] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5168.816343] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5174.841062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5178.190234] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5188.064103] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5188.069057] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5188.080843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5189.603597] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5189.606485] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5189.619636] Lustre: Skipped 2 previous similar messages [ 5189.626485] Lustre: Skipped 3 previous similar messages [ 5191.367889] Lustre: server umount lustre-MDT0000 complete [ 5193.339808] LustreError: 227906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692461 with bad export cookie 3390682596442835530 [ 5193.344172] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5193.351653] LustreError: 227906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5193.514900] Lustre: server umount lustre-MDT0001 complete [ 5205.980625] Lustre: server umount lustre-OST0000 complete [ 5217.501825] Lustre: server umount lustre-OST0001 complete [ 5229.457726] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5235.437614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5235.879185] LustreError: 230239:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5235.896142] LustreError: 230239:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5235.942950] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5238.949905] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5241.313841] LustreError: 230240:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5244.740552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5244.975711] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5247.521607] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5249.340574] Lustre: 231344:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5253.024417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5255.269445] LustreError: 231698:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5256.676344] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5260.395731] LustreError: 231699:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5260.410485] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 5260.412826] LustreError: 231699:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5262.247625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5262.452422] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5262.459000] Lustre: Skipped 1 previous similar message [ 5266.385965] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5271.405316] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5273.688654] Lustre: 233181:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5280.739588] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5280.747774] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.757263] LustreError: 230954:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5280.789589] Lustre: server umount mylustre-MDT0000 complete [ 5282.791118] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5282.805383] Lustre: Skipped 1 previous similar message [ 5283.156624] LustreError: 233381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692551 with bad export cookie 3390682596442838323 [ 5283.161317] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5283.163582] LustreError: 233381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5285.459912] Lustre: server umount mylustre-MDT0001 complete [ 5298.202569] Lustre: server umount mylustre-OST0000 complete [ 5310.765474] Lustre: server umount mylustre-OST0001 complete [ 5323.061720] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5328.755461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5329.042970] LustreError: 235471:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5329.054111] LustreError: 235471:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 5329.103637] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5331.265455] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5335.883172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5338.355919] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5339.830292] Lustre: 236578:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5343.640957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5345.894405] LustreError: 236932:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5345.903790] LustreError: 236932:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5346.928829] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5351.551860] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5353.777934] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 5353.779416] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 5354.503893] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5358.698475] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5360.460301] Lustre: 238414:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5364.196524] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5364.198596] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5364.206030] Lustre: Skipped 3 previous similar messages [ 5364.212212] Lustre: Skipped 3 previous similar messages [ 5369.314763] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5369.321034] Lustre: Skipped 3 previous similar messages [ 5370.134990] Lustre: server umount tfs-MDT0000 complete [ 5371.941040] LustreError: 235446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692640 with bad export cookie 3390682596442839982 [ 5371.943111] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5371.947623] LustreError: 235446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5372.107334] Lustre: server umount tfs-MDT0001 complete [ 5383.709853] Lustre: server umount tfs-OST0000 complete [ 5395.993843] Lustre: server umount tfs-OST0001 complete [ 5407.278218] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5412.266439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5412.493532] LustreError: 240643:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5412.505417] LustreError: 240643:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5412.540218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5412.545187] Lustre: Skipped 3 previous similar messages [ 5414.368369] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5418.813121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5421.335871] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5422.964497] Lustre: 241749:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5426.319139] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5429.344924] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5433.645571] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5436.521438] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5437.868728] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 5437.874898] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 5446.555904] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5448.728877] Lustre: 243588:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5452.658774] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:05:19 (1779692719) [ 5468.644314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5468.648133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5468.651858] Lustre: Skipped 3 previous similar messages [ 5468.658066] Lustre: Skipped 1 previous similar message [ 5471.696434] Lustre: server umount lustre-MDT0000 complete [ 5473.561343] LustreError: 240619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692742 with bad export cookie 3390682596442841655 [ 5473.562871] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5473.567750] LustreError: 240619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5478.881262] LustreError: 240638:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5478.881646] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5478.891546] LustreError: 240638:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5478.893559] Lustre: Skipped 5 previous similar messages [ 5479.756677] Lustre: server umount lustre-MDT0001 complete [ 5487.694952] Lustre: server umount lustre-OST0000 complete [ 5495.818221] Lustre: server umount lustre-OST0001 complete [ 5499.272207] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 5504.587837] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5523.003250] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5523.003250] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5523.003250] [ 5523.038119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5523.163891] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5523.184400] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5523.230418] Lustre: lustre-MDT0000: new disk, initializing [ 5523.294152] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5525.245942] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5530.816178] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 5530.816178] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5530.816178] [ 5530.833381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5530.898408] Lustre: 246736:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5530.934831] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5530.938614] Lustre: Skipped 1 previous similar message [ 5530.982709] Lustre: lustre-MDT0001: new disk, initializing [ 5531.047536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5531.054606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5532.896603] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5538.154813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5538.278626] Lustre: lustre-OST0000: new disk, initializing [ 5538.281247] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5538.285420] Lustre: 248285:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5539.685889] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5539.692810] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5539.761566] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5541.117068] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5546.688179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5546.767627] Lustre: lustre-OST0001: new disk, initializing [ 5546.770590] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5546.774961] Lustre: 249137:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5546.818337] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5546.821895] Lustre: Skipped 6 previous similar messages [ 5548.814644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5548.819642] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5548.870754] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5549.480484] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5555.561670] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:07:02 (1779692822) [ 5562.699596] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:07:10 (1779692830) [ 5575.648449] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5575.654309] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5575.660425] Lustre: Skipped 3 previous similar messages [ 5575.665482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5575.669231] Lustre: Skipped 1 previous similar message [ 5581.004294] Lustre: server umount lustre-OST0000 complete [ 5589.150359] Lustre: server umount lustre-MDT0000 complete [ 5591.373465] LustreError: 246728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779692860 with bad export cookie 3390682596442843209 [ 5591.379349] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5591.385629] LustreError: 246728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5597.600855] Lustre: server umount lustre-MDT0001 complete [ 5602.915072] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 5614.497355] Lustre: server umount lustre-OST0001 complete [ 5617.201903] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 5624.610368] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5641.948517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5642.106819] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5642.129095] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5642.184141] Lustre: lustre-MDT0000: new disk, initializing [ 5642.248702] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5644.848354] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5657.527500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5657.724133] Lustre: 253597:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5658.149247] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5664.407784] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5679.130426] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5689.423822] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5703.567573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5704.337574] Lustre: lustre-OST0000: new disk, initializing [ 5704.355653] Lustre: Skipped 1 previous similar message [ 5704.383773] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5704.393523] Lustre: Skipped 2 previous similar messages [ 5704.407370] Lustre: 255340:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5705.917654] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5705.929296] Lustre: Skipped 1 previous similar message [ 5705.933958] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5706.063746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5714.278072] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5722.810513] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5731.810200] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5731.810665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5731.818440] LustreError: Skipped 2 previous similar messages [ 5731.820438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5731.820444] Lustre: Skipped 7 previous similar messages [ 5731.885367] Lustre: Skipped 6 previous similar messages [ 5736.682169] Lustre: server umount lustre-OST0000 complete [ 5748.121368] Lustre: server umount lustre-MDT0000 complete [ 5752.290731] LustreError: 253607:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5752.316041] LustreError: 253607:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 5753.066024] LustreError: 253588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779693021 with bad export cookie 3390682596442844672 [ 5753.074689] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5753.084677] LustreError: 253588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5753.559351] Lustre: server umount lustre-MDT0001 complete [ 5767.650846] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 5771.318231] Key type lgssc unregistered [ 5771.725422] LNet: 256973:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5771.730200] LNetError: 256973:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5771.753178] LNet: Removed LNI 192.168.203.113@tcp [ 5772.956169] Key type .llcrypt unregistered [ 5772.962066] Key type ._llcrypt unregistered [ 5797.959816] Key type ._llcrypt registered [ 5797.962340] Key type .llcrypt registered [ 5798.064082] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5798.933646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5798.960720] alg: No test for adler32 (adler32-zlib) [ 5799.965355] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 5800.190413] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 5801.879187] Key type lgssc registered [ 5803.221965] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5810.563346] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5810.582319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5811.988587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5816.330221] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5821.423735] LustreError: 258442:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5821.443162] LustreError: 258442:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5824.141461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5824.490833] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5827.999475] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5832.980878] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5836.122571] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5841.936339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5842.273691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5847.660114] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5852.237868] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5853.042588] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5856.940218] LustreError: 260037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5856.951311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5857.037660] Lustre: server umount lustre-OST0000 complete [ 5862.370653] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5862.381508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5862.389369] Lustre: Skipped 1 previous similar message [ 5866.900422] Lustre: server umount lustre-MDT0000 complete [ 5867.495845] LustreError: 259647:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5867.503929] LustreError: 259647:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5869.861781] LustreError: 260632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779693138 with bad export cookie 10259252849936839165 [ 5869.863799] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5869.876210] LustreError: 260632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5870.112481] Lustre: server umount lustre-MDT0001 complete [ 5878.862633] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 5881.691381] Key type lgssc unregistered [ 5882.012816] LNet: 261669:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5882.025234] LNetError: 261669:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5882.038544] LNet: Removed LNI 192.168.203.113@tcp [ 5882.783700] Key type .llcrypt unregistered [ 5882.790325] Key type ._llcrypt unregistered [ 5898.741638] Key type ._llcrypt registered [ 5898.744233] Key type .llcrypt registered [ 5898.845345] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 5899.805116] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5899.883620] alg: No test for adler32 (adler32-zlib) [ 5900.981358] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 5901.179884] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 5902.857091] Key type lgssc registered [ 5903.936325] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5912.661800] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5912.680602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5928.223653] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5933.343371] LustreError: 262846:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.113@tcp: failed processing log, type 4: rc = -110 [ 5960.095192] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5966.260486] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 5972.284941] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5974.577274] Lustre: server umount lustre-OST0000 complete [ 5990.335166] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 5993.810707] Key type lgssc unregistered [ 5994.355900] LNet: 264346:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5994.378926] LNetError: 264346:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5995.430345] LNet: Removed LNI 192.168.203.113@tcp [ 5996.736600] Key type .llcrypt unregistered [ 5996.741490] Key type ._llcrypt unregistered [ 6041.772739] Key type ._llcrypt registered [ 6041.775456] Key type .llcrypt registered [ 6041.871759] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 6042.860372] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6042.879387] alg: No test for adler32 (adler32-zlib) [ 6044.065433] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 6044.320470] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 6046.007811] Key type lgssc registered [ 6046.847311] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6058.431133] loop0: detected capacity change from 0 to 204800000 [ 6058.666888] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6058.690686] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6059.178741] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6060.452561] loop0: detected capacity change from 0 to 204800000 [ 6060.650217] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6060.665707] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6060.907283] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6061.906782] loop0: detected capacity change from 0 to 204800000 [ 6062.113721] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6062.128342] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6062.308838] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6063.768360] loop0: detected capacity change from 0 to 204800000 [ 6064.123381] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6064.148584] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6064.251116] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6065.239436] loop0: detected capacity change from 0 to 204800000 [ 6065.258884] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6069.377601] loop0: detected capacity change from 0 to 204800000 [ 6069.421215] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6073.330801] loop0: detected capacity change from 0 to 204800000 [ 6073.351807] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6077.075423] loop0: detected capacity change from 0 to 204800000 [ 6077.124192] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6080.501409] loop0: detected capacity change from 0 to 204800000 [ 6080.602664] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6080.662438] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6080.800066] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6080.820297] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6080.830702] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6080.841710] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6080.862509] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6080.882495] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6080.895727] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6080.905491] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6080.915574] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6080.924763] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6080.932308] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6080.941916] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6080.980844] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6081.008825] Lustre: 267744:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6086.398281] Lustre: server umount MGS complete [ 6087.184670] loop0: detected capacity change from 0 to 204800000 [ 6087.264929] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6087.791052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6087.870243] Lustre: lustre-MDT0000: Aborting client recovery [ 6088.724469] loop1: detected capacity change from 0 to 204800000 [ 6088.838763] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6088.916148] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6088.926378] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6088.935121] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6088.942202] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6088.951265] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6088.958593] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6088.964758] Lustre: 268177:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6088.988912] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6089.174401] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6089.240854] Lustre: lustre-MDT0001: Aborting client recovery [ 6090.080100] loop2: detected capacity change from 0 to 204800000 [ 6090.252713] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6090.306567] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6090.317328] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6090.334719] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6090.347968] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6090.357537] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6090.372796] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6090.390233] Lustre: 268258:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6090.516772] Lustre: Found index 0 for lustre-OST0000, updating log [ 6090.551652] Lustre: 268277:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6090.633692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6090.669388] Lustre: lustre-OST0000: Aborting recovery [ 6091.581628] loop3: detected capacity change from 0 to 204800000 [ 6091.736535] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6091.830589] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6091.847043] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6091.862474] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6091.877651] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6091.888156] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6091.897717] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6091.908585] Lustre: 268343:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6091.929562] Lustre: Found index 1 for lustre-OST0001, updating log [ 6091.982468] Lustre: 268349:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6092.110067] Lustre: lustre-OST0001: Aborting recovery [ 6097.429351] LustreError: 268120:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 6098.422186] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6098.422209] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6098.425930] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6098.434724] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6098.472982] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6098.498536] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6099.546556] LustreError: 268087:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 793us: rc = -2 [ 6100.059557] LustreError: 268088:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 725us: rc = -2 [ 6100.075785] LustreError: 268088:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 6113.766872] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6113.784568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6113.788289] Lustre: Skipped 2 previous similar messages [ 6113.797258] Lustre: Skipped 3 previous similar messages [ 6118.883345] LustreError: 268085:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6118.904980] LustreError: 268085:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 6119.154549] Lustre: server umount lustre-MDT0000 complete [ 6120.305386] LustreError: 268070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779693388 with bad export cookie 12675611564678140194 [ 6120.323907] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6124.002035] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6124.003748] LustreError: 268086:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6124.023456] Lustre: Skipped 1 previous similar message [ 6124.027434] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6124.070203] LustreError: 268086:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6125.033319] LustreError: 268085:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6125.033807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6125.080342] Lustre: Skipped 2 previous similar messages [ 6127.126786] Lustre: server umount lustre-MDT0001 complete [ 6134.676787] Lustre: server umount lustre-OST0000 complete [ 6150.114064] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6150.463486] Lustre: server umount lustre-OST0001 complete [ 6158.791422] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:17:05 (1779693425) [ 6187.493949] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 6197.510934] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 6230.023750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6230.280261] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6230.317394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6230.451123] Lustre: lustre-MDT0000: new disk, initializing [ 6230.570804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6230.577175] Lustre: Skipped 1 previous similar message [ 6230.603269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6234.355147] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6243.147626] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6243.233506] Lustre: 272385:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6243.312969] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6243.316845] Lustre: Skipped 1 previous similar message [ 6243.400507] Lustre: lustre-MDT0001: new disk, initializing [ 6243.497197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6243.555574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6243.562204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6247.126765] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6253.656995] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6257.556568] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6264.302701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6264.533363] Lustre: lustre-OST0000: new disk, initializing [ 6264.536581] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6264.542354] Lustre: 274125:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6264.628079] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6270.732527] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6272.564447] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6272.576649] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6272.637492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6277.622082] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6283.076886] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6294.360777] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6301.929770] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6354.912776] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6354.929161] LustreError: Skipped 1 previous similar message [ 6354.929180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6354.942031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6354.948730] Lustre: Skipped 2 previous similar messages [ 6354.955239] Lustre: Skipped 1 previous similar message [ 6359.954080] Lustre: server umount lustre-OST0000 complete [ 6365.161080] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6365.164123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6365.176238] Lustre: Skipped 1 previous similar message [ 6370.273239] LustreError: 272396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6370.288366] LustreError: 272396:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6370.330387] Lustre: server umount lustre-MDT0000 complete [ 6373.796927] LustreError: 273161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779693642 with bad export cookie 12675611564678142917 [ 6373.797512] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6373.813289] LustreError: 273161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6374.187433] Lustre: server umount lustre-MDT0001 complete [ 6384.037451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6388.304460] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6395.117819] Lustre: server umount MGS complete [ 6406.417638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6406.902677] LustreError: 277929:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6406.940516] LustreError: 277942:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6407.044022] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6410.764367] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6417.380241] LustreError: 277929:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6417.396653] LustreError: 277929:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6418.905093] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6423.630617] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6428.785069] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6432.630422] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6439.943174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6440.301970] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6440.313059] Lustre: Skipped 1 previous similar message [ 6446.787466] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6452.957370] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6485.476101] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6485.494945] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6485.519237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6485.533421] Lustre: Skipped 1 previous similar message [ 6488.544559] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6489.990984] Lustre: server umount lustre-OST0000 complete [ 6496.225790] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6496.237128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6496.256536] Lustre: Skipped 1 previous similar message [ 6500.533261] Lustre: server umount lustre-MDT0000 complete [ 6503.912329] LustreError: 277926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6503.943158] LustreError: 277926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6504.460266] LustreError: 280315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779693773 with bad export cookie 12675611564678144744 [ 6504.461532] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6504.477762] LustreError: 280315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6504.921870] Lustre: server umount lustre-MDT0001 complete [ 6516.779358] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 6519.774564] Key type lgssc unregistered [ 6520.035502] LNet: 281519:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6520.043017] LNetError: 281519:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6521.061852] LNet: Removed LNI 192.168.203.113@tcp [ 6521.911175] Key type .llcrypt unregistered [ 6521.914657] Key type ._llcrypt unregistered [ 6559.307648] Key type ._llcrypt registered [ 6559.309762] Key type .llcrypt registered [ 6559.431668] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 6573.037422] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 6574.210687] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6574.277053] alg: No test for adler32 (adler32-zlib) [ 6575.294411] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 6575.524425] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 6577.220633] Key type lgssc registered [ 6578.208647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6613.213662] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6613.230516] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6614.554387] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6614.602931] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6614.665253] Lustre: lustre-MDT0000: new disk, initializing [ 6614.734679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6614.749836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6620.421896] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6630.743697] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6630.823681] Lustre: 285441:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6630.873206] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6630.876185] Lustre: Skipped 1 previous similar message [ 6630.924446] Lustre: lustre-MDT0001: new disk, initializing [ 6631.024053] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6631.051110] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6631.062384] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6634.929631] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6642.805286] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6647.333461] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6654.279208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6654.627810] Lustre: lustre-OST0000: new disk, initializing [ 6654.632976] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6654.646204] Lustre: 287184:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6654.737607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6660.194082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6660.202541] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6660.368053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6660.967156] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6668.533708] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6674.460534] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6685.653426] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6694.657897] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6704.534738] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6742.503894] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6742.513768] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6742.522237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6742.554217] Lustre: Skipped 1 previous similar message [ 6747.632887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6747.640814] Lustre: Skipped 2 previous similar messages [ 6752.751144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6756.321917] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6756.469268] Lustre: server umount lustre-OST0000 complete [ 6762.976886] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6762.986925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6763.016670] Lustre: Skipped 1 previous similar message [ 6763.040884] Lustre: Skipped 2 previous similar messages [ 6768.359168] Lustre: server umount lustre-MDT0000 complete [ 6773.127305] LustreError: 288616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779694041 with bad export cookie 6911523063867726981 [ 6773.132820] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6773.136774] LustreError: 288616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6773.355962] Lustre: server umount lustre-MDT0001 complete [ 6781.499300] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6786.790388] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6791.446997] Lustre: server umount MGS complete [ 6801.579137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6802.118368] LustreError: 291023:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6802.135790] LustreError: 291023:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6802.152990] LustreError: 291036:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6802.230213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6806.187331] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6807.525569] LustreError: 291024:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6811.625395] LustreError: 291023:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6813.960220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6814.406761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6819.187391] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6824.198806] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6827.413879] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6833.571704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6833.825918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6839.681946] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 6845.270553] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6878.178806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6878.182266] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6878.192034] LustreError: Skipped 1 previous similar message [ 6878.196418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6878.232447] Lustre: Skipped 1 previous similar message [ 6883.641433] Lustre: server umount lustre-OST0000 complete [ 6888.422672] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6888.429266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6888.446728] Lustre: Skipped 1 previous similar message [ 6888.461938] Lustre: Skipped 4 previous similar messages [ 6893.617870] Lustre: server umount lustre-MDT0000 complete [ 6896.861825] LustreError: 293407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779694165 with bad export cookie 6911523063867728927 [ 6896.871584] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6896.873957] LustreError: 293407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6897.204112] Lustre: server umount lustre-MDT0001 complete [ 6906.320407] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 6908.836315] Key type lgssc unregistered [ 6909.139080] LNet: 294606:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6909.162946] LNetError: 294606:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6909.176748] LNet: Removed LNI 192.168.203.113@tcp [ 6910.061231] Key type .llcrypt unregistered [ 6910.070059] Key type ._llcrypt unregistered [ 6929.272448] Key type ._llcrypt registered [ 6929.274794] Key type .llcrypt registered [ 6929.401835] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 6943.116794] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 6944.051107] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6944.095516] alg: No test for adler32 (adler32-zlib) [ 6945.281087] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 6945.525384] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 6947.249094] Key type lgssc registered [ 6948.625363] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6972.945915] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 6987.317608] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6987.350322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6988.770566] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6988.836423] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6988.938445] Lustre: lustre-MDT0000: new disk, initializing [ 6989.037042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6989.057798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6993.426369] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 7003.131116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7003.347145] Lustre: lustre-OST0000: new disk, initializing [ 7003.351410] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7003.357241] Lustre: Skipped 1 previous similar message [ 7003.366834] Lustre: 298590:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7003.448594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7009.371910] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 7011.418249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7011.426340] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7011.460982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7815.141698] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7815.159754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7817.240380] Lustre: server umount lustre-MDT0000 complete [ 7820.774594] LustreError: 297792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695089 with bad export cookie 17966249056324108877 [ 7820.779987] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7820.782137] LustreError: 297792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7820.957355] Lustre: server umount lustre-OST0000 complete [ 7845.532402] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 7857.829629] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 7895.930897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7896.214316] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7896.260851] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7896.332935] Lustre: lustre-MDT0000: new disk, initializing [ 7896.403849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7896.418436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7900.988270] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 7911.489996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7911.604742] Lustre: 302870:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7911.668405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7911.675574] Lustre: Skipped 1 previous similar message [ 7911.752840] Lustre: lustre-MDT0001: new disk, initializing [ 7911.887840] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7911.940422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7911.958354] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7915.008938] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 7923.461744] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7927.992272] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7935.522801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7935.991918] Lustre: lustre-OST0000: new disk, initializing [ 7935.995524] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7936.002486] Lustre: 304611:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7936.145390] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7942.761993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7942.769682] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7942.919666] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7943.562062] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 7951.232848] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7955.913434] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7956.147425] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7961.155049] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7961.393279] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7963.617022] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7963.621803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7963.640659] Lustre: Skipped 1 previous similar message [ 7968.740544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7968.745595] Lustre: Skipped 2 previous similar messages [ 7969.461285] Lustre: server umount lustre-OST0000 complete [ 7973.864519] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7973.906121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7978.464262] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7978.469779] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7978.488074] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7979.885476] Lustre: server umount lustre-MDT0000 complete [ 7983.803648] LustreError: 302861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695252 with bad export cookie 17966249056327049696 [ 7983.811051] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7983.828132] LustreError: 302861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7984.330133] Lustre: server umount lustre-MDT0001 complete [ 7987.764552] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7989.603836] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:47:36 (1779695256) [ 7996.221298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7996.670956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8000.855850] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8002.026210] LustreError: 306779:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8002.054241] LustreError: 306779:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8007.138078] LustreError: 306778:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8009.833807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8010.350357] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8014.855645] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8019.714476] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8022.612143] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8029.122735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8029.558738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8037.540359] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8044.377604] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8052.238439] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8052.427113] Lustre: lustre-OST0001: new disk, initializing [ 8052.431317] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8052.438358] Lustre: 309275:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8052.535872] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8057.590501] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8062.019286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8062.028168] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8062.074386] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8062.083089] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8063.140188] LustreError: 309832:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8063.165878] LustreError: 309832:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8064.176296] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8066.278682] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8066.308364] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 8066.506799] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8066.711967] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8070.441232] LustreError: 309832:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8070.450476] LustreError: 309832:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 8070.585531] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ce5448d0000.ost_server_uuid 50 [ 8072.166364] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ce5448d0000.ost_server_uuid in FULL state after 0 sec [ 8076.317685] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ce5448d0000.ost_server_uuid 50 [ 8077.856593] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9ce5448d0000.ost_server_uuid in FULL state after 0 sec [ 8078.641874] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8078.655354] LustreError: 309828:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [ 8079.737054] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8093.160850] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8093.171391] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8093.175750] Lustre: Skipped 2 previous similar messages [ 8098.930642] Lustre: server umount lustre-OST0001 complete [ 8103.394669] LustreError: 310580:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8103.421079] LustreError: 310580:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8106.465610] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8106.485190] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8106.506894] Lustre: Skipped 1 previous similar message [ 8108.521687] LustreError: 308879:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8108.523035] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8108.546967] LustreError: 308879:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8112.161094] Lustre: server umount lustre-OST0000 complete [ 8117.728283] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8117.735199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8117.737687] Lustre: Skipped 7 previous similar messages [ 8122.168609] Lustre: server umount lustre-MDT0000 complete [ 8123.876626] LustreError: 307490:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8123.894737] LustreError: 307490:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8125.498781] LustreError: 306758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695394 with bad export cookie 17966249056327050578 [ 8125.509831] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8125.513465] LustreError: 306758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8125.999146] Lustre: server umount lustre-MDT0001 complete [ 8136.360576] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 8138.752328] Key type lgssc unregistered [ 8139.029930] LNet: 311719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8139.035197] LNetError: 311719:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8140.072546] LNet: Removed LNI 192.168.203.113@tcp [ 8140.866462] Key type .llcrypt unregistered [ 8140.868472] Key type ._llcrypt unregistered [ 8163.902629] Key type ._llcrypt registered [ 8163.906827] Key type .llcrypt registered [ 8164.015645] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 8165.024380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8165.089249] alg: No test for adler32 (adler32-zlib) [ 8166.073743] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 8166.302330] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 8168.003630] Key type lgssc registered [ 8169.105372] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8176.187753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8176.223526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8177.651890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8181.808219] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8186.797693] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8187.878221] LustreError: 313127:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8187.906370] LustreError: 313127:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8192.991865] LustreError: 313128:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8193.190596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8193.607260] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8197.477684] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8206.001813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8206.429353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8211.553945] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8216.189728] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8218.728560] LustreError: 314679:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8222.297937] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8222.534656] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8223.558692] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8223.575316] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 8228.540680] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8233.426740] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 8240.015778] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 8240.192632] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8341.474812] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8341.478638] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8341.501634] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8341.504056] Lustre: Skipped 1 previous similar message [ 8345.295507] Lustre: server umount lustre-OST0001 complete [ 8346.593561] LustreError: 314677:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8346.632141] LustreError: 314677:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8351.718209] LustreError: 315630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8351.719518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8351.731454] LustreError: 315630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8351.731817] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8351.732863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8351.747082] Lustre: Skipped 2 previous similar messages [ 8356.834228] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8356.845340] Lustre: Skipped 2 previous similar messages [ 8361.954401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8361.957812] LustreError: 315630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8361.969179] Lustre: Skipped 1 previous similar message [ 8361.980476] LustreError: 315630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8365.023256] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8365.176439] Lustre: server umount lustre-OST0000 complete [ 8372.192283] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8372.194914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8372.204524] Lustre: Skipped 1 previous similar message [ 8372.227246] Lustre: Skipped 1 previous similar message [ 8374.615574] Lustre: server umount lustre-MDT0000 complete [ 8377.541112] LustreError: 313108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695646 with bad export cookie 3153923453960155097 [ 8377.543424] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8377.554054] LustreError: 313108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8377.915108] Lustre: server umount lustre-MDT0001 complete [ 8388.441947] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 8391.132632] Key type lgssc unregistered [ 8391.456545] LNet: 317683:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8391.474563] LNetError: 317683:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8391.488867] LNet: Removed LNI 192.168.203.113@tcp [ 8392.479073] Key type .llcrypt unregistered [ 8392.483631] Key type ._llcrypt unregistered [ 8436.723047] Key type ._llcrypt registered [ 8436.724992] Key type .llcrypt registered [ 8436.809604] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 8437.776044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8437.835033] alg: No test for adler32 (adler32-zlib) [ 8438.905855] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 8439.154222] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 8440.834082] Key type lgssc registered [ 8441.955682] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8449.930716] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8449.942461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8451.442361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8455.262816] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8461.794374] LustreError: 319956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8461.812321] LustreError: 319956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8462.251994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8462.513713] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8465.521638] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8467.834392] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8473.546105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8473.877424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8476.974863] LustreError: 321414:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8477.011781] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 8479.406417] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8482.276643] LustreError: 321415:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8486.680103] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8486.897356] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8487.939779] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 8492.746677] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8501.292061] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8505.392567] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8518.634540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8518.642835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8518.645943] Lustre: Skipped 2 previous similar messages [ 8518.660220] Lustre: Skipped 3 previous similar messages [ 8521.034740] Lustre: server umount lustre-MDT0000 complete [ 8523.763959] LustreError: 321065:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8523.787932] LustreError: 321065:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8524.045541] LustreError: 319938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695792 with bad export cookie 4494022376764495779 [ 8524.051201] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8524.056201] LustreError: 319938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8524.425445] Lustre: server umount lustre-MDT0001 complete [ 8538.157862] Lustre: server umount lustre-OST0000 complete [ 8552.092554] Lustre: server umount lustre-OST0001 complete [ 8565.429654] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 8574.383843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8574.804396] LustreError: 325137:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8574.873520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8578.514888] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8585.189758] LustreError: 325137:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8585.204334] LustreError: 325137:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8585.264631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8585.640193] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8589.455889] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8591.914765] Lustre: 326243:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8597.813287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8600.127581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 8603.614543] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8605.162856] LustreError: 326598:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8605.189230] LustreError: 326598:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8611.397543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8611.630764] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8611.634976] Lustre: Skipped 1 previous similar message [ 8616.949155] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8617.158927] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8624.179675] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8627.352094] Lustre: 328080:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8631.783149] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8631.790598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8631.803832] Lustre: Skipped 1 previous similar message [ 8631.812226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8637.018337] Lustre: server umount lustre-MDT0000 complete [ 8637.412792] LustreError: 326620:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8637.438334] LustreError: 326620:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 8640.095625] LustreError: 325118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779695908 with bad export cookie 4494022376764497480 [ 8640.112039] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8640.113312] LustreError: 325118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8640.494984] Lustre: server umount lustre-MDT0001 complete [ 8654.366410] Lustre: server umount lustre-OST0000 complete [ 8666.588948] Lustre: server umount lustre-OST0001 complete [ 8669.761910] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 8671.079045] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 8672.593281] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:58:59 (1779695939) [ 8691.440575] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 8699.887689] loop0: detected capacity change from 0 to 1099511627776 [ 8700.585481] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 8700.592243] *** VALIDATE xfs *** [ 8700.609750] XFS (loop0): Mounting V5 Filesystem [ 8700.675749] XFS (loop0): Ending clean mount [ 8701.965108] loop1: detected capacity change from 0 to 18691697672192 [ 8802.429510] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [ 8813.046438] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 8845.581270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8845.751300] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8845.792211] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8845.872235] Lustre: lustre-MDT0000: new disk, initializing [ 8845.931402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8845.944067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8849.222618] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8857.491910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8857.592736] Lustre: 333173:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8857.637644] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8857.641307] Lustre: Skipped 1 previous similar message [ 8857.699727] Lustre: lustre-MDT0001: new disk, initializing [ 8857.827423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8857.841158] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8861.477245] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8867.719537] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8870.735124] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8876.683340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8876.954486] Lustre: lustre-OST0000: new disk, initializing [ 8876.960626] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8876.968205] Lustre: 334917:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8878.773044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8878.789553] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8878.941451] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8883.419695] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8891.720752] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8897.338732] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8897.525844] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8902.322417] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8902.506840] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8904.675256] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8904.678955] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8904.682047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8904.682054] Lustre: Skipped 4 previous similar messages [ 8904.699508] Lustre: Skipped 4 previous similar messages [ 8909.798525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8909.809449] Lustre: Skipped 1 previous similar message [ 8910.641679] Lustre: server umount lustre-OST0000 complete [ 8914.411354] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8914.425248] LustreError: Skipped 1 previous similar message [ 8914.431532] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8914.447276] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8914.456811] Lustre: Skipped 1 previous similar message [ 8919.860718] Lustre: server umount lustre-MDT0000 complete [ 8920.041835] LustreError: 334365:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8920.063318] LustreError: 334365:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8923.513711] LustreError: 333166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779696192 with bad export cookie 4494022376764499034 [ 8923.515891] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8923.539382] LustreError: 333166:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8923.936559] Lustre: server umount lustre-MDT0001 complete [ 8930.713716] XFS (loop0): Unmounting Filesystem [ 8934.201759] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:03:21 (1779696201) [ 8940.790491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8941.385119] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8941.397448] Lustre: Skipped 2 previous similar messages [ 8945.650240] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8953.730475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8958.684197] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8964.030615] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8967.932924] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8974.077872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8980.448848] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 8985.543950] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8994.277498] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8994.286854] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8994.300654] Lustre: Skipped 1 previous similar message [ 8994.303823] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8994.306146] Lustre: Skipped 1 previous similar message [ 8999.523645] Lustre: server umount lustre-OST0000 complete [ 9005.546912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9005.553878] LustreError: Skipped 1 previous similar message [ 9005.556457] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9005.562953] Lustre: Skipped 1 previous similar message [ 9009.056859] Lustre: server umount lustre-MDT0000 complete [ 9013.066925] LustreError: 339661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779696281 with bad export cookie 4494022376764499916 [ 9013.081426] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9013.091996] LustreError: 339661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9013.576953] Lustre: server umount lustre-MDT0001 complete [ 9025.859248] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [ 9028.457208] Key type lgssc unregistered [ 9028.746287] LNet: 340599:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9028.760626] LNetError: 340599:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9028.781641] LNet: Removed LNI 192.168.203.113@tcp [ 9029.478117] Key type .llcrypt unregistered [ 9029.479935] Key type ._llcrypt unregistered [ 9051.508953] Key type ._llcrypt registered [ 9051.511976] Key type .llcrypt registered [ 9051.601714] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [ 9052.845142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9052.862963] alg: No test for adler32 (adler32-zlib) [ 9053.914915] Lustre: Lustre: Build Version: 2.17.53_24_g1d9adb4 [ 9054.120802] LNet: Added LNI 192.168.203.113@tcp [8/256/0/180] [ 9055.783203] Key type lgssc registered [ 9056.794758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9063.184709] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9063.229501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9064.717569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9068.467655] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9075.172241] LustreError: 342128:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9075.196891] LustreError: 342128:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9076.737255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9077.195315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9081.330511] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9085.821966] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9089.488225] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9096.377353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9096.808813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9103.028631] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9108.478565] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9120.369137] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.13@tcp (stopping) [ 9120.741981] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9120.767646] Lustre: Skipped 1 previous similar message [ 9125.453883] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.13@tcp (stopping) [ 9125.462375] Lustre: Skipped 2 previous similar messages [ 9125.739483] Lustre: server umount lustre-MDT0001 complete [ 9125.862164] LustreError: 342128:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9125.881481] LustreError: 342128:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9130.574248] LustreError: 342123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9135.555393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9135.676463] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9135.681646] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9135.695656] LustreError: 342125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9135.728428] LustreError: 342125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9135.755214] Lustre: 342116:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9136.013618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9139.877963] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9141.232748] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9144.317347] LustreError: 345375:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9144.329117] LustreError: 345375:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9144.338507] LustreError: 342120:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9144.414846] LustreError: 345378:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.113@tcp: cfg command failed: rc = -17 [ 9144.429346] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9144.429346] [ 9448.040979] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.13@tcp (stopping) [ 9448.424625] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9448.438162] Lustre: Skipped 1 previous similar message [ 9451.268216] Lustre: server umount lustre-MDT0001 complete [ 9453.133272] LustreError: 347403:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9460.075589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9460.165433] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9460.171406] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9460.204513] Lustre: 342118:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9460.434029] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9464.391663] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9465.842489] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9466.352560] LustreError: 354157:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9466.355591] LustreError: 354157:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9466.359043] LustreError: 342120:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9466.365476] LustreError: 342120:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9466.414473] LustreError: 354166:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 9466.420128] LustreError: 354166:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 9466.423883] LustreError: 354166:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.113@tcp: cfg command failed: rc = -17 [ 9466.434312] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.113@tcp [ 9466.434312] [ 9770.609185] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.13@tcp (stopping) [ 9770.619204] Lustre: Skipped 2 previous similar messages [ 9773.043288] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9773.049716] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9775.694475] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.13@tcp (stopping) [ 9775.699159] Lustre: Skipped 2 previous similar messages [ 9784.287628] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9784.645497] Lustre: server umount lustre-MDT0001 complete [ 9785.942526] LustreError: 342123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9785.960614] LustreError: 342123:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 9794.190249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9794.247594] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9794.251565] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9794.279725] Lustre: 342116:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9794.592178] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 9798.817428] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [ 9799.660357] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9800.184527] LustreError: 363109:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9800.198491] LustreError: 363109:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9800.212639] LustreError: 342120:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9800.217423] LustreError: 342120:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9800.293080] LustreError: 363117:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 9800.304576] LustreError: 363117:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.113@tcp: cfg command failed: rc = -114 [ 9800.311884] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 9800.311884] [10122.210153] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10122.210331] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10122.237377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10122.241980] Lustre: Skipped 2 previous similar messages [10122.258857] Lustre: Skipped 7 previous similar messages [10126.146883] Lustre: server umount lustre-MDT0000 complete [10127.337747] LustreError: 342124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10127.362871] LustreError: 342124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [10129.943369] LustreError: 342110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779697398 with bad export cookie 5610392651803951441 [10129.949312] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10129.955224] LustreError: 342110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10132.449815] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10132.464057] Lustre: Skipped 1 previous similar message [10136.286994] Lustre: server umount lustre-MDT0001 complete [10140.155244] Lustre: server umount lustre-OST0000 complete [10149.608715] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [10163.179766] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [10199.579832] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:24:26 (1779697466) [10207.258732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10207.578226] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10207.620458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10207.723997] Lustre: lustre-MDT0000: new disk, initializing [10207.826311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10207.851540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10212.499215] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [10225.618357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10225.845383] Lustre: 374142:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10225.992290] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10226.003755] Lustre: Skipped 1 previous similar message [10226.203903] Lustre: lustre-MDT0001: new disk, initializing [10226.337198] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10226.390061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10226.416718] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10232.178633] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [10240.942123] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10244.556393] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10251.651843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10251.990265] Lustre: lustre-OST0000: new disk, initializing [10251.994176] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10252.000039] Lustre: 375882:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10252.080557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10258.444604] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_default_debug -1 all [10260.045826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10260.061383] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10260.230473] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10266.448358] Lustre: DEBUG MARKER: oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10275.810401] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10275.813429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10275.819240] Lustre: Skipped 1 previous similar message [10275.830867] Lustre: Skipped 4 previous similar messages [10276.714953] Lustre: server umount lustre-MDT0000 complete [10280.451213] LustreError: 374134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779697549 with bad export cookie 5610392651803953394 [10280.453115] LustreError: 374930:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10280.462942] LustreError: MGC192.168.203.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10280.469677] LustreError: 374134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10280.931148] LustreError: 374148:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10280.955320] LustreError: 374148:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10281.103215] Lustre: server umount lustre-MDT0001 complete [10287.368724] Lustre: server umount lustre-OST0000 complete [10318.509528] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing set_hostid [10330.563480] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing load_modules_local [10364.480370] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10073 sec ========= 04:27:11 (1779697631) [10366.152987] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:27:12 (1779697632) === [10369.979073] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:27:16 (1779697636) === [10410.417963] Lustre: DEBUG MARKER: oleg313-server.virtnet: executing unload_modules_local [10413.588676] Key type lgssc unregistered [10413.986850] LNet: 382082:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10413.993949] LNetError: 382082:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10414.016264] LNet: Removed LNI 192.168.203.113@tcp [10414.916113] Key type .llcrypt unregistered [10414.921462] Key type ._llcrypt unregistered