[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 546983475 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002302] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.006980] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008007] pid_max: default: 32768 minimum: 301 [ 0.009117] LSM: Security Framework initializing [ 0.010048] Yama: becoming mindful. [ 0.011038] SELinux: Initializing. [ 0.012072] *** VALIDATE selinux *** [ 0.020825] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025684] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026174] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027125] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028140] *** VALIDATE tmpfs *** [ 0.029430] *** VALIDATE proc *** [ 0.031223] *** VALIDATE cgroup *** [ 0.032009] *** VALIDATE cgroup2 *** [ 0.033239] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034145] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036044] Spectre V2 : User space: Vulnerable [ 0.037010] Speculative Store Bypass: Vulnerable [ 0.040453] debug: unmapping init [mem 0xffffffffb3659000-0xffffffffb3660fff] [ 0.042202] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043787] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044030] ... version: 2 [ 0.045010] ... bit width: 48 [ 0.046015] ... generic registers: 4 [ 0.047015] ... value mask: 0000ffffffffffff [ 0.048014] ... max period: 00007fffffffffff [ 0.049011] ... fixed-purpose events: 3 [ 0.049772] ... event mask: 000000070000000f [ 0.050275] rcu: Hierarchical SRCU implementation. [ 0.052297] smp: Bringing up secondary CPUs ... [ 0.053482] x86: Booting SMP configuration: [ 0.054016] .... node #0, CPUs: #1 #2 #3 [ 0.057184] smp: Brought up 1 node, 4 CPUs [ 0.059016] smpboot: Max logical packages: 1 [ 0.060019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.137285] node 0 deferred pages initialised in 75ms [ 0.142144] devtmpfs: initialized [ 0.143250] x86/mm: Memory block size: 128MB [ 0.145872] gcov: version magic: 0x41383552 [ 0.147351] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.151114] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.154376] pinctrl core: initialized pinctrl subsystem [ 0.156199] [ 0.156774] ************************************************************* [ 0.160015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162012] ** ** [ 0.165015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.167013] ** ** [ 0.169017] ** This means that this kernel is built to expose internal ** [ 0.171016] ** IOMMU data structures, which may compromise security on ** [ 0.172007] ** your system. ** [ 0.174009] ** ** [ 0.175007] ** If you see this message and you are not debugging the ** [ 0.177012] ** kernel, report this immediately to your vendor! ** [ 0.179016] ** ** [ 0.182015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.184015] ************************************************************* [ 0.187691] NET: Registered protocol family 16 [ 0.189463] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.192075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.194093] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200085] cpuidle: using governor menu [ 0.201883] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.202500] PCI: Using configuration type 1 for base access [ 0.203155] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.211116] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.214020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.217159] cryptd: max_cpu_qlen set to 1000 [ 0.222226] ACPI: Added _OSI(Module Device) [ 0.224036] ACPI: Added _OSI(Processor Device) [ 0.226022] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.228029] ACPI: Added _OSI(Processor Aggregator Device) [ 0.233553] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.242011] ACPI: Interpreter enabled [ 0.243079] ACPI: PM: (supports S0 S3 S4 S5) [ 0.244010] ACPI: Using IOAPIC for interrupt routing [ 0.246108] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.250436] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.261097] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.263063] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.266025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.270087] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.276498] acpiphp: Slot [2] registered [ 0.278154] acpiphp: Slot [5] registered [ 0.279133] acpiphp: Slot [6] registered [ 0.281175] acpiphp: Slot [7] registered [ 0.282159] acpiphp: Slot [8] registered [ 0.284162] acpiphp: Slot [9] registered [ 0.286155] acpiphp: Slot [10] registered [ 0.287156] acpiphp: Slot [3] registered [ 0.289132] acpiphp: Slot [4] registered [ 0.290126] acpiphp: Slot [11] registered [ 0.292117] acpiphp: Slot [12] registered [ 0.293155] acpiphp: Slot [13] registered [ 0.294100] acpiphp: Slot [14] registered [ 0.296125] acpiphp: Slot [15] registered [ 0.297133] acpiphp: Slot [16] registered [ 0.299108] acpiphp: Slot [17] registered [ 0.300096] acpiphp: Slot [18] registered [ 0.302107] acpiphp: Slot [19] registered [ 0.303159] acpiphp: Slot [20] registered [ 0.305113] acpiphp: Slot [21] registered [ 0.306134] acpiphp: Slot [22] registered [ 0.308122] acpiphp: Slot [23] registered [ 0.310148] acpiphp: Slot [24] registered [ 0.311134] acpiphp: Slot [25] registered [ 0.313122] acpiphp: Slot [26] registered [ 0.314114] acpiphp: Slot [27] registered [ 0.316138] acpiphp: Slot [28] registered [ 0.317115] acpiphp: Slot [29] registered [ 0.319237] acpiphp: Slot [30] registered [ 0.321159] acpiphp: Slot [31] registered [ 0.323082] PCI host bridge to bus 0000:00 [ 0.324022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.327042] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.329034] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.332037] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.335027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.337020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.339173] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.342201] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.345311] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.355014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.360058] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.363033] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.365017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.367017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.370827] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.373874] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.377048] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.380950] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.387019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.402000] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.407846] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.412881] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.432021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.438025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.456026] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.468941] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.477027] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.485024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.504029] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.518210] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.525024] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.532027] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.548030] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.563174] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.571027] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.579029] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.595031] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.608796] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.618027] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.624026] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.643026] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.656870] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.674030] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.689026] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.706030] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.720000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.722480] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.727559] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.730518] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.733311] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.740095] iommu: Default domain type: Passthrough [ 0.742576] SCSI subsystem initialized [ 0.744252] ACPI: bus type USB registered [ 0.746183] usbcore: registered new interface driver usbfs [ 0.748122] usbcore: registered new interface driver hub [ 0.749120] usbcore: registered new device driver usb [ 0.752271] pps_core: LinuxPPS API ver. 1 registered [ 0.754015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.758097] PTP clock support registered [ 0.760082] EDAC MC: Ver: 3.0.0 [ 0.762184] PCI: Using ACPI for IRQ routing [ 0.764948] NetLabel: Initializing [ 0.766022] NetLabel: domain hash size = 128 [ 0.767015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.769090] NetLabel: unlabeled traffic allowed by default [ 0.771165] vgaarb: loaded [ 0.773284] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.775031] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.781373] clocksource: Switched to clocksource kvm-clock [ 0.893688] VFS: Disk quotas dquot_6.6.0 [ 0.895255] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.897694] *** VALIDATE ramfs *** [ 0.898979] *** VALIDATE hugetlbfs *** [ 0.900644] pnp: PnP ACPI init [ 0.903106] pnp: PnP ACPI: found 6 devices [ 0.923089] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.926538] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.928715] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.930874] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.933375] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.935758] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.938614] NET: Registered protocol family 2 [ 0.940975] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.945857] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.949570] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.954953] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.958625] TCP: Hash tables configured (established 65536 bind 65536) [ 0.961702] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.964104] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.966031] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.968317] NET: Registered protocol family 1 [ 0.970896] RPC: Registered named UNIX socket transport module. [ 0.973300] RPC: Registered udp transport module. [ 0.975036] RPC: Registered tcp transport module. [ 0.976876] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.979390] NET: Registered protocol family 44 [ 0.981030] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.983388] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.985606] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.988076] PCI: CLS 0 bytes, default 64 [ 0.989814] Unpacking initramfs... [ 2.422424] debug: unmapping init [mem 0xffff89df7cc54000-0xffff89df7ffbffff] [ 2.426820] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.428962] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.431757] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.944483] Initialise system trusted keyrings [ 2.948141] Key type blacklist registered [ 2.950203] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.958913] zbud: loaded [ 2.962330] *** VALIDATE nfs *** [ 2.963581] *** VALIDATE nfs4 *** [ 2.964969] pstore: using deflate compression [ 2.968414] Platform Keyring initialized [ 3.087374] NET: Registered protocol family 38 [ 3.088697] Key type asymmetric registered [ 3.090132] Asymmetric key parser 'x509' registered [ 3.091540] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.093975] io scheduler mq-deadline registered [ 3.094968] io scheduler kyber registered [ 3.096137] io scheduler bfq registered [ 3.098250] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.100204] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.101832] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.103746] ACPI: Power Button [PWRF] [ 3.108329] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.114648] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.126614] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.133273] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.152132] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.181354] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.212332] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.218646] Non-volatile memory driver v1.3 [ 3.220542] Linux agpgart interface v0.103 [ 3.248866] virtio_blk virtio1: [vda] 146136 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.251731] vda: detected capacity change from 0 to 74821632 [ 3.273555] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.276847] vdb: detected capacity change from 0 to 1073741824 [ 3.298982] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.300954] vdc: detected capacity change from 0 to 2621440000 [ 3.320529] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.322639] vdd: detected capacity change from 0 to 2621440000 [ 3.339602] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.342658] vde: detected capacity change from 0 to 4294967296 [ 3.360732] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.363819] vdf: detected capacity change from 0 to 4294967296 [ 3.391886] libphy: Fixed MDIO Bus: probed [ 3.398510] usbcore: registered new interface driver usbserial_generic [ 3.401233] usbserial: USB Serial support registered for generic [ 3.403232] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.407523] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.409335] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.412391] mousedev: PS/2 mouse device common for all mice [ 3.415637] rtc_cmos 00:05: RTC can wake from S4 [ 3.419360] rtc_cmos 00:05: registered as rtc0 [ 3.421260] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.424804] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.427787] intel_pstate: CPU model not supported [ 3.431076] hid: raw HID events driver (C) Jiri Kosina [ 3.434290] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.435035] usbcore: registered new interface driver usbhid [ 3.439843] usbhid: USB HID core driver [ 3.442291] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.442626] drop_monitor: Initializing network drop monitor service [ 3.447951] Initializing XFRM netlink socket [ 3.450455] NET: Registered protocol family 10 [ 3.454616] Segment Routing with IPv6 [ 3.455968] NET: Registered protocol family 17 [ 3.459137] mpls_gso: MPLS GSO support [ 3.469281] RAS: Correctable Errors collector initialized. [ 3.471171] AVX version of gcm_enc/dec engaged. [ 3.472560] AES CTR mode by8 optimization enabled [ 3.558911] sched_clock: Marking stable (3558891245, 0)->(4448016938, -889125693) [ 3.562034] registered taskstats version 1 [ 3.565163] Loading compiled-in X.509 certificates [ 3.566904] zswap: loaded using pool lzo/zbud [ 3.595125] Key type big_key registered [ 3.613376] Key type encrypted registered [ 3.615269] ima: No TPM chip found, activating TPM-bypass! [ 3.617483] ima: Allocated hash algorithm: sha1 [ 3.619271] ima: No architecture policies found [ 3.621079] evm: Initialising EVM extended attributes: [ 3.623072] evm: security.selinux [ 3.624357] evm: security.ima [ 3.625571] evm: security.capability [ 3.627021] evm: HMAC attrs: 0x1 [ 3.630484] rtc_cmos 00:05: setting system clock to 2026-08-23 04:57:00 UTC (1787461020) [ 3.637083] debug: unmapping init [mem 0xffffffffb4603000-0xffffffffb47fffff] [ 3.642036] debug: unmapping init [mem 0xffffffffb3382000-0xffffffffb3658fff] [ 3.652082] Write protecting the kernel read-only data: 28672k [ 3.654460] debug: unmapping init [mem 0xffffffffb1a03000-0xffffffffb1bfffff] [ 3.657086] debug: unmapping init [mem 0xffffffffb2314000-0xffffffffb23fffff] [ 3.693902] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.702488] systemd[1]: Detected virtualization kvm. [ 3.704639] systemd[1]: Detected architecture x86-64. [ 3.706688] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.738992] systemd[1]: No hostname configured. [ 3.741054] systemd[1]: Set hostname to . [ 3.742941] random: systemd: uninitialized urandom read (16 bytes read) [ 3.745553] systemd[1]: Initializing machine ID from random generator. [ 3.789718] random: ln: uninitialized urandom read (6 bytes read) [ 3.867319] random: systemd: uninitialized urandom read (16 bytes read) [ 3.870390] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.875353] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.880270] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Swap. [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.465753] device-mapper: uevent: version 1.0.3 [ 4.468377] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.119896] virtio_net virtio0 ens2: renamed from eth0 [ 5.133685] random: fast init done [ 5.202023] scsi host0: ata_piix [ 5.208503] scsi host1: ata_piix [ 5.210061] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.211977] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.572677] dracut-initqueue[589]: RTNETLINK answers: File exists [ 10.141672] random: crng init done [ 10.143376] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.480278] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.503523] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.736315] SELinux: Disabled at runtime. [ 11.797224] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.806512] systemd[1]: Detected virtualization kvm. [ 11.807964] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.194309] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.197534] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.202499] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.208365] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.211909] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.219177] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.234261] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ 12.328129] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.628263] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.899781] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.949271] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.074825] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.086800] EDAC sbridge: Ver: 1.1.2 [ 14.680365] Key type dns_resolver registered [ 14.987508] NFS: Registering the id_resolver key type [ 14.989474] Key type id_resolver registered [ 14.991259] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg152-server login: [ 42.109864] libcfs: loading out-of-tree module taints kernel. [ 42.126951] Key type ._llcrypt registered [ 42.128344] Key type .llcrypt registered [ 42.188058] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 51.258996] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 51.881858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 51.888604] alg: No test for adler32 (adler32-zlib) [ 52.924518] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 53.292056] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 54.928252] Key type lgssc registered [ 55.733241] Lustre: Echo OBD driver; http://www.lustre.org/ [ 75.246815] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 90.633019] hrtimer: interrupt took 5292993 ns [ 117.734455] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 131.579977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 131.645322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 132.987281] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 133.036260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 133.191925] Lustre: lustre-MDT0000: new disk, initializing [ 133.317927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 133.350308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 138.449341] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 153.633850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 153.760638] Lustre: 6501:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 153.792186] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 153.796061] Lustre: Skipped 1 previous similar message [ 153.867161] Lustre: lustre-MDT0001: new disk, initializing [ 153.927243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 153.948338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 153.962755] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 159.301113] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 164.258321] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 174.486312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 174.878149] Lustre: lustre-OST0000: new disk, initializing [ 174.881689] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 174.888367] Lustre: 8441:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 174.955979] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 181.595727] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 183.874606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 183.894277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 183.976609] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 197.268507] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 197.447362] Lustre: lustre-OST0001: new disk, initializing [ 197.450077] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 197.457352] Lustre: 9513:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 197.537841] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 203.874381] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.377866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 207.396033] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 207.501475] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 216.254247] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 221.701987] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 228.197861] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing check_logdir /tmp/testlogs/ [ 233.853132] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing yml_node [ 239.187974] Lustre: DEBUG MARKER: Client: 2.17.57.65 [ 242.181268] Lustre: DEBUG MARKER: MDS: 2.17.57.65 [ 245.500329] Lustre: DEBUG MARKER: OSS: 2.17.57.65 [ 247.373775] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Aug 23 01:01:01 EDT 2026 [ 263.892881] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 265.077185] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 282.080906] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 282.106304] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 282.126430] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 284.142362] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 284.143628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 284.176421] Lustre: Skipped 3 previous similar messages [ 289.250796] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 289.264620] Lustre: Skipped 3 previous similar messages [ 294.375173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 294.378028] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 294.391310] Lustre: Skipped 3 previous similar messages [ 294.697262] Lustre: server umount lustre-MDT0000 complete [ 298.537521] LustreError: 6492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787461315 with bad export cookie 4954510206309835530 [ 298.541627] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 298.545853] LustreError: 6492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 299.489668] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 299.498203] Lustre: Skipped 3 previous similar messages [ 299.504306] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 299.523444] Lustre: Skipped 1 previous similar message [ 304.999300] Lustre: server umount lustre-MDT0001 complete [ 309.587957] Lustre: server umount lustre-OST0000 complete [ 314.034470] Lustre: server umount lustre-OST0001 complete [ 321.010820] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 332.380805] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 372.330975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 372.588504] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 372.614799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 372.698512] Lustre: lustre-MDT0000: new disk, initializing [ 372.753214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 372.765855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 377.068309] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 387.611442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 387.841310] Lustre: 15080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 387.937106] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 387.948260] Lustre: Skipped 1 previous similar message [ 388.093566] Lustre: lustre-MDT0001: new disk, initializing [ 388.235376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 388.263377] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 388.276705] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 393.243608] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 402.532128] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 407.996191] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 414.169303] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 414.585856] Lustre: lustre-OST0000: new disk, initializing [ 414.594462] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 414.604973] Lustre: 17047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 414.716955] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 416.091424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 416.114398] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 416.276164] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 422.390303] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 433.271166] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 438.425446] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 438.729862] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 444.285343] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 444.532093] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 446.945793] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 446.949308] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 446.964187] LustreError: Skipped 1 previous similar message [ 446.965109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 446.965114] Lustre: Skipped 2 previous similar messages [ 447.022343] Lustre: Skipped 1 previous similar message [ 452.708632] Lustre: server umount lustre-OST0000 complete [ 459.745771] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 459.748612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 462.801542] Lustre: server umount lustre-MDT0000 complete [ 465.383693] LustreError: 15088:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 465.398405] LustreError: 15088:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 466.767835] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787461483 with bad export cookie 4954510206309837063 [ 466.771636] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 466.780159] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 467.259333] Lustre: server umount lustre-MDT0001 complete [ 472.248929] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:04:46 (1787461486) [ 479.170971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.574607] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.712149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 484.538763] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 484.833250] LustreError: 19315:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 489.960978] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 494.127350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 499.199426] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 507.605905] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 513.575818] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 520.370483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 526.287443] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 534.330101] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 541.584473] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 550.369594] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 550.386560] Lustre: Skipped 2 previous similar messages [ 550.393336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 550.406549] Lustre: Skipped 8 previous similar messages [ 554.076298] Lustre: server umount lustre-MDT0000 complete [ 555.526943] LustreError: 19315:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 555.542383] LustreError: 19315:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 558.051468] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787461574 with bad export cookie 4954510206309837945 [ 558.055678] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 558.063326] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 558.426556] Lustre: server umount lustre-MDT0001 complete [ 573.189574] Lustre: server umount lustre-OST0000 complete [ 595.987243] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 607.104909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 607.700571] LustreError: 23694:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 607.783755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 607.801017] Lustre: Skipped 2 previous similar messages [ 612.546468] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 621.199548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 626.638561] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 629.955948] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 636.676890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 643.798646] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 653.072151] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 653.257611] Lustre: lustre-OST0001: new disk, initializing [ 653.265531] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 653.270738] Lustre: 26008:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 655.053867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 655.068413] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 655.164928] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 660.254055] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 670.472079] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 674.583597] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 688.098522] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 688.102955] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 688.115468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 693.627654] Lustre: server umount lustre-MDT0000 complete [ 696.293792] LustreError: 23695:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 696.314188] LustreError: 23695:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 698.393828] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787461715 with bad export cookie 4954510206309838792 [ 698.402895] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 698.407533] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 698.887700] Lustre: server umount lustre-MDT0001 complete [ 713.623446] Lustre: server umount lustre-OST0000 complete [ 726.434880] Lustre: server umount lustre-OST0001 complete [ 743.556380] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 755.019630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 755.399236] LustreError: 29031:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 755.471183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 755.480731] Lustre: Skipped 3 previous similar messages [ 760.253762] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 771.622932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 777.656533] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 781.402577] Lustre: 30179:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 788.930599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 796.759177] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 807.258935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 814.405608] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 823.857688] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 828.145718] Lustre: 32057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 838.631364] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 838.655431] Lustre: Skipped 6 previous similar messages [ 838.659363] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 838.666240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 838.674645] Lustre: Skipped 7 previous similar messages [ 841.402348] Lustre: server umount lustre-MDT0000 complete [ 843.745788] LustreError: 29767:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 843.776129] LustreError: 29767:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 845.570376] LustreError: 30561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787461862 with bad export cookie 4954510206309840087 [ 845.572845] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 845.586436] LustreError: 30561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 846.105314] Lustre: server umount lustre-MDT0001 complete [ 860.836559] Lustre: server umount lustre-OST0000 complete [ 864.736157] Lustre: 3635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787461865/real 1787461865] req@ffff89dfec4f9500 x1874288779231488/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787461881 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 865.034386] Lustre: server umount lustre-OST0001 complete [ 874.944036] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:11:29 (1787461889) [ 903.090315] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 913.315627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 918.672774] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 928.444855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 934.550910] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 937.858613] Lustre: 36009:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 944.982478] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 950.584751] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 961.050276] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 969.859231] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 978.890793] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 983.161925] Lustre: 37890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 990.694200] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 990.716775] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 990.744154] Lustre: Skipped 1 previous similar message [ 990.754268] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 996.074284] Lustre: server umount lustre-MDT0000 complete [ 997.351756] LustreError: 35610:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 997.379522] LustreError: 35610:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 999.877580] LustreError: 34841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787462016 with bad export cookie 4954510206309841193 [ 999.885598] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 999.888631] LustreError: 34841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1000.267191] Lustre: server umount lustre-MDT0001 complete [ 1014.955279] Lustre: server umount lustre-OST0000 complete [ 1029.067636] Lustre: server umount lustre-OST0001 complete [ 1039.841814] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:14:13 (1787462053) [ 1059.760444] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1070.704420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1071.216057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1071.231097] Lustre: Skipped 7 previous similar messages [ 1075.939154] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1085.060237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1089.575446] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1091.840756] Lustre: 41181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1099.673712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1107.611534] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1117.089525] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1124.686667] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1133.358793] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1138.040759] Lustre: 43056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1140.002310] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1140.009127] Lustre: Skipped 1 previous similar message [ 1146.849456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1146.878074] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1146.900962] Lustre: Skipped 3 previous similar messages [ 1150.092777] Lustre: server umount lustre-MDT0000 complete [ 1153.660702] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787462170 with bad export cookie 4954510206309842292 [ 1153.667272] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1153.681586] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1160.109307] Lustre: server umount lustre-MDT0001 complete [ 1168.301797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1185.154800] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1188.860512] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1195.155560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1198.670220] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1198.689429] Lustre: Skipped 1 previous similar message [ 1199.517335] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1206.751710] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1208.531160] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1215.250560] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1216.956311] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1228.399818] Lustre: server umount lustre-MDT0000 complete [ 1232.388839] LustreError: 43058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787462249 with bad export cookie 4954510206309843489 [ 1232.391671] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1232.394373] LustreError: 43058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1232.731459] Lustre: server umount lustre-MDT0001 complete [ 1247.314226] Lustre: server umount lustre-OST0000 complete [ 1250.273652] Lustre: 3635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787462251/real 1787462251] req@ffff89dfff3eaa00 x1874288779377664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787462267 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1251.010570] Lustre: server umount lustre-OST0001 complete [ 1260.287570] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:17:54 (1787462274) [ 1279.012529] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1288.296376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1288.584435] LustreError: 47670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1288.595710] LustreError: 47670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 22 previous similar messages [ 1292.896299] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1301.399047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1307.140372] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1310.704814] Lustre: 48822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1318.017794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1324.913720] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1332.826357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1339.541927] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1347.429762] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1351.700711] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1363.428428] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1363.443975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1363.452908] Lustre: Skipped 17 previous similar messages [ 1364.484554] Lustre: server umount lustre-MDT0000 complete [ 1368.548313] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787462385 with bad export cookie 4954510206309844280 [ 1368.556315] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1368.567212] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1368.895585] Lustre: server umount lustre-MDT0001 complete [ 1383.269556] Lustre: server umount lustre-OST0000 complete [ 1385.184280] Lustre: 3636:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787462386/real 1787462386] req@ffff89dec1842300 x1874288779432576/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787462402 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1387.557948] Lustre: server umount lustre-OST0001 complete [ 1399.004618] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:20:12 (1787462412) [ 1402.664864] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1404.832693] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:20:18 (1787462418) [ 1418.763204] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1430.105171] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1453.266759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1453.514777] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1453.565928] Lustre: lustre-MDT0000: new disk, initializing [ 1453.626345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1459.866496] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1474.148295] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1484.493741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1484.732072] Lustre: lustre-OST0000: new disk, initializing [ 1484.741324] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1484.748227] Lustre: Skipped 1 previous similar message [ 1484.753438] Lustre: 54895:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1486.433167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1486.441869] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1486.485079] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1490.564649] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1501.039692] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1503.790245] Lustre: DEBUG MARKER: create test files [ 1542.631262] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1542.637526] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1542.643495] Lustre: Skipped 15 previous similar messages [ 1546.044388] Lustre: server umount lustre-OST0000 complete [ 1551.045360] Lustre: server umount lustre-MDT0000 complete [ 1565.051332] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1568.328211] Key type lgssc unregistered [ 1568.621141] LNet: 56527:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1568.626771] LNetError: 56527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1568.654103] LNet: Removed LNI 192.168.201.152@tcp [ 1569.663913] Key type .llcrypt unregistered [ 1569.666074] Key type ._llcrypt unregistered [ 1599.093455] Key type ._llcrypt registered [ 1599.102974] Key type .llcrypt registered [ 1599.243343] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1600.983950] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1601.088128] alg: No test for adler32 (adler32-zlib) [ 1602.243452] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 1602.508886] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1604.224197] Key type lgssc registered [ 1605.165830] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1611.934655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1611.974865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1613.490225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1618.221737] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1626.129841] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1632.745371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1633.024547] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1639.167623] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1639.310054] Lustre: 59264:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1639.317603] Lustre: 59264:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1639.322660] Lustre: 59264:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1639.327560] Lustre: 59264:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1639.332957] Lustre: 59264:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1639.339118] Lustre: 59264:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1639.605859] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1647.599984] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1653.083745] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1658.622235] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1670.118645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1670.121831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1670.136896] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1672.525594] Lustre: server umount lustre-OST0000 complete [ 1676.920373] Lustre: server umount lustre-MDT0000 complete [ 1688.403399] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1690.538231] Key type lgssc unregistered [ 1690.838677] LNet: 60878:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1690.851432] LNetError: 60878:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1690.894389] LNet: Removed LNI 192.168.201.152@tcp [ 1691.747173] Key type .llcrypt unregistered [ 1691.750222] Key type ._llcrypt unregistered [ 1722.212902] Key type ._llcrypt registered [ 1722.215522] Key type .llcrypt registered [ 1722.329189] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1723.472942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1723.578579] alg: No test for adler32 (adler32-zlib) [ 1724.685910] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 1724.900742] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1726.552169] Key type lgssc registered [ 1727.636881] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1735.238165] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1735.253760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1736.697440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1741.551538] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1749.823703] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1756.894769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1757.249070] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1761.339463] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1761.479276] Lustre: 63719:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1761.510330] Lustre: 63719:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1761.527490] Lustre: 63719:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1761.531789] Lustre: 63719:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1761.535091] Lustre: 63719:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1761.538448] Lustre: 63719:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1764.000205] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1772.374288] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1778.196183] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1791.977594] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1791.985410] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1792.020355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1796.186977] Lustre: server umount lustre-OST0000 complete [ 1801.801867] Lustre: server umount lustre-MDT0000 complete [ 1816.637467] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1820.343712] Key type lgssc unregistered [ 1820.663975] LNet: 65277:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1820.675437] LNetError: 65277:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1820.691815] LNet: Removed LNI 192.168.201.152@tcp [ 1821.765644] Key type .llcrypt unregistered [ 1821.768148] Key type ._llcrypt unregistered [ 1843.793800] Key type ._llcrypt registered [ 1843.796796] Key type .llcrypt registered [ 1843.997543] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1862.159617] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1864.008291] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1864.100476] alg: No test for adler32 (adler32-zlib) [ 1865.211967] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 1865.682276] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1867.440658] Key type lgssc registered [ 1868.889479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1915.836126] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1915.893169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1917.231443] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1917.264413] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1917.384613] Lustre: lustre-MDT0000: new disk, initializing [ 1917.633973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1917.668987] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1922.689988] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1934.552513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1934.672776] Lustre: 68432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1934.735723] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1934.742371] Lustre: Skipped 1 previous similar message [ 1934.840566] Lustre: lustre-MDT0001: new disk, initializing [ 1934.955454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1935.050646] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1935.058182] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1940.067743] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1950.704586] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1956.667532] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1963.644291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1963.954148] Lustre: lustre-OST0000: new disk, initializing [ 1963.958574] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1963.967906] Lustre: 70400:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1964.083761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1970.969755] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1973.319616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1973.345674] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1973.443757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1980.418664] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1985.920971] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1986.243272] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1991.860828] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1992.194487] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1999.334152] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1999.337200] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1999.356482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1999.377451] Lustre: Skipped 1 previous similar message [ 1999.406021] Lustre: Skipped 1 previous similar message [ 2000.810950] Lustre: server umount lustre-OST0000 complete [ 2006.498027] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2006.521611] LustreError: Skipped 1 previous similar message [ 2006.529911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2006.542292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2009.574602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2009.577087] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.670944] Lustre: server umount lustre-MDT0000 complete [ 2016.165989] LustreError: 68424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787463033 with bad export cookie 1718502083734452219 [ 2016.176809] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2016.189938] LustreError: 68424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2016.938892] Lustre: server umount lustre-MDT0001 complete [ 2027.442261] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:30:41 (1787463041) [ 2038.728232] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2084.635688] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 2096.670718] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2136.186046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2136.690800] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2136.755631] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2136.926711] Lustre: lustre-MDT0000: new disk, initializing [ 2137.039107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2137.079275] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2142.788890] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2155.639854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2155.785530] Lustre: 76433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2155.842784] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2155.854140] Lustre: Skipped 1 previous similar message [ 2155.942692] Lustre: lustre-MDT0001: new disk, initializing [ 2156.093217] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2156.158884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2156.172826] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2162.077511] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2172.737477] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2179.990446] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2188.329128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2188.722248] Lustre: lustre-OST0000: new disk, initializing [ 2188.731540] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2188.747534] Lustre: 78403:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2188.882237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2195.287893] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2198.608738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2198.615564] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2198.650634] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2206.968376] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2214.738757] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2215.021899] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2220.482591] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2220.743643] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2224.619814] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2224.621114] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2224.635161] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2224.640574] Lustre: Skipped 2 previous similar messages [ 2229.740879] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2229.754349] Lustre: Skipped 1 previous similar message [ 2237.408135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2237.654479] Lustre: server umount lustre-OST0000 complete [ 2243.046374] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2243.050818] LustreError: Skipped 1 previous similar message [ 2243.054577] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2243.068439] Lustre: Skipped 1 previous similar message [ 2243.072627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2243.077754] Lustre: Skipped 2 previous similar messages [ 2247.903842] Lustre: server umount lustre-MDT0000 complete [ 2250.227864] LustreError: 76446:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2250.249719] LustreError: 76446:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2251.327300] LustreError: 76424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787463268 with bad export cookie 1718502083734453101 [ 2251.328937] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2251.337763] LustreError: 76424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2251.649849] Lustre: server umount lustre-MDT0001 complete [ 2259.528776] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:34:33 (1787463273) [ 2265.340907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2265.610672] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2265.645866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2269.473391] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2270.692889] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2275.825866] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2280.929793] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2281.241279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2288.193042] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2297.411726] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2304.966570] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2313.007531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2313.262920] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2313.268136] Lustre: Skipped 1 previous similar message [ 2319.976893] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2328.801595] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2332.982207] Lustre: 83489:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 957e18ff-35ca-4e9a-a292-f6670dc3b818 at adminstrative request [ 2335.793459] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2335.814810] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2345.964836] Lustre: MGS: Client 957e18ff-35ca-4e9a-a292-f6670dc3b818 (at 0@lo) reconnecting [ 2345.975632] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x17d9588a46c2bedf to 0x17d9588a46c2c17f [ 2345.997602] Lustre: MGC192.168.201.152@tcp: Connection restored to 0@lo (at 0@lo) [ 2371.327247] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2371.576974] Lustre: lustre-OST0001: new disk, initializing [ 2371.595359] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2371.602828] Lustre: 83883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2378.422059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2378.451903] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2378.602814] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2378.785974] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2389.405500] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2394.106776] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2394.124049] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2394.134150] Lustre: Skipped 1 previous similar message [ 2394.139171] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2394.175034] Lustre: Skipped 3 previous similar messages [ 2407.393047] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2407.483605] Lustre: server umount lustre-OST0001 complete [ 2409.461280] LustreError: 82641:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2409.494211] LustreError: 82641:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2412.514098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2412.520334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2412.535346] Lustre: Skipped 1 previous similar message [ 2426.336157] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2426.645602] Lustre: server umount lustre-OST0000 complete [ 2435.050604] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2435.057270] Lustre: Skipped 12 previous similar messages [ 2437.612141] Lustre: server umount lustre-MDT0000 complete [ 2440.161934] LustreError: 81545:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2440.176619] LustreError: 81545:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 2441.723120] LustreError: 83882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787463458 with bad export cookie 1718502083734454655 [ 2441.727863] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2441.741035] LustreError: 83882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2442.259908] Lustre: server umount lustre-MDT0001 complete [ 2451.913450] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:37:46 (1787463466) [ 2453.853821] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2455.798980] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:37:50 (1787463470) [ 2457.502601] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2459.562468] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:37:53 (1787463473) [ 2461.235610] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2463.260859] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:37:57 (1787463477) [ 2465.237799] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2467.846753] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:38:01 (1787463481) [ 2469.690335] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2476.382354] loop: module loaded [ 2476.404692] loop0: detected capacity change from 0 to 4294967296 [ 2476.462365] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2478.284811] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2482.149356] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2487.254190] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2518.988437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2519.052602] LustreError: 88010:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2519.062672] LustreError: 88010:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2519.074648] LustreError: 88010:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2519.085164] LustreError: 88010:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2541.780441] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 2554.618949] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2594.659269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2594.973922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2595.011212] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2595.095832] Lustre: lustre-MDT0000: new disk, initializing [ 2595.252218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2595.262852] Lustre: Skipped 1 previous similar message [ 2595.284708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2601.047938] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2615.473543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2615.646292] Lustre: 90878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2615.972035] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2622.696994] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2635.559747] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2645.414885] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2657.110663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2657.551943] Lustre: 92849:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2665.096371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2665.110770] Lustre: Skipped 1 previous similar message [ 2665.122537] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2665.228773] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2665.246825] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2676.846768] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2684.650968] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2685.157213] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2693.708810] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2694.146243] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2701.282813] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2701.286138] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2701.297444] LustreError: Skipped 2 previous similar messages [ 2701.303651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2701.323749] Lustre: Skipped 4 previous similar messages [ 2701.361432] Lustre: Skipped 2 previous similar messages [ 2703.945910] Lustre: server umount lustre-OST0000 complete [ 2723.296175] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2723.536625] Lustre: server umount lustre-MDT0000 complete [ 2726.888065] LustreError: 92250:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2726.902737] LustreError: 92250:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2727.602265] LustreError: 90868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787463744 with bad export cookie 1718502083734455271 [ 2727.619275] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2728.209968] Lustre: server umount lustre-MDT0001 complete [ 2738.253513] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:42:32 (1787463752) [ 2745.774841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2746.348792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2746.363890] Lustre: Skipped 2 previous similar messages [ 2751.314217] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2759.665944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2763.871812] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2775.947663] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2783.921794] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2793.112373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2803.316792] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2813.611556] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2822.626772] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2823.056062] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2832.102341] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2832.794664] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2842.053611] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2842.300253] Lustre: lustre-OST0001: new disk, initializing [ 2842.303091] Lustre: Skipped 2 previous similar messages [ 2842.306931] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2842.312099] Lustre: Skipped 3 previous similar messages [ 2842.320310] Lustre: 98642:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2849.665314] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2850.987333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2851.017954] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2851.288274] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2861.807300] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2867.856544] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2868.122403] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2873.627326] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2873.879588] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2895.935705] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2919.778547] Lustre: Failing over lustre-MDT0000 [ 2920.778267] Lustre: server umount lustre-MDT0000 complete [ 2922.674524] LustreError: 96001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2922.703531] LustreError: 96001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2922.995267] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2923.024535] Lustre: Skipped 2 previous similar messages [ 2938.341229] Lustre: 66655:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787463939/real 1787463939] req@00000000e76f3d81 x1874290679261952/t0(0) o400->MGC192.168.201.152@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787463955 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2938.372993] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2949.759712] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2949.761502] LDISKFS-fs (dm-0): recovery complete [ 2949.766219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2963.937918] LustreError: 66652:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000084961f1b x1874290679272960/t0(0) o250->MGC192.168.201.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2964.439782] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2968.657305] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2968.662478] Lustre: 95263:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2969.606730] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2969.610544] Lustre: 95267:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2969.625542] Lustre: 95267:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2969.756260] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2970.075105] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2970.414583] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2970.431996] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2970.492326] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2970.767182] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2970.779632] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2970.796597] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2970.813874] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2971.457137] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2971.464282] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2971.798930] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2971.810895] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2973.115176] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2973.133044] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2973.457028] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2973.470180] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2974.112135] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2974.122222] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2977.322736] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2977.339989] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2977.669913] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2977.690969] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2978.329866] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2978.343136] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2985.325957] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2985.354508] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2985.690218] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2985.707357] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2986.353297] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2986.369355] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3001.610547] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3001.626324] Lustre: 101504:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 3001.953574] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3001.960640] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3002.600125] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3002.603857] LustreError: 101504:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3028.035067] Lustre: 101504:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3028.045956] Lustre: 101504:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3028.052751] Lustre: 101504:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0551ad17-7e74-47bb-94b4-05706a876f08@192.168.201.52@tcp [ 3028.066650] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3028.074782] Lustre: 101504:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000001a562e88 x1874290659105280/t0(8589934779) o101->0551ad17-7e74-47bb-94b4-05706a876f08@192.168.201.52@tcp:45/0 lens 592/0 e 0 to 0 dl 1787464055 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3028.112149] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3028.121420] Lustre: lustre-MDT0000: Denying connection for new client 0551ad17-7e74-47bb-94b4-05706a876f08 (at 192.168.201.52@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3028.179234] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3028.241254] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3028.254936] Lustre: Skipped 2 previous similar messages [ 3028.257105] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3028.320574] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3028.320791] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3044.324215] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3044.335463] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3044.344600] Lustre: Skipped 6 previous similar messages [ 3049.800470] Lustre: server umount lustre-OST0000 complete [ 3051.505776] LustreError: 97090:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3051.530934] LustreError: 97090:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 55 previous similar messages [ 3054.565110] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3054.570309] Lustre: Skipped 5 previous similar messages [ 3068.384424] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3068.569676] Lustre: server umount lustre-OST0001 complete [ 3080.935615] Lustre: server umount lustre-MDT0000 complete [ 3086.074855] LustreError: 98641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787464102 with bad export cookie 1718502083734506490 [ 3086.092149] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3086.093622] LustreError: 98641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3086.828877] Lustre: server umount lustre-MDT0001 complete [ 3098.450178] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:48:33 (1787464113) [ 3108.060982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3108.065686] Lustre: *** cfs_fail_loc=197, val=0*** [ 3123.814601] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3128.992699] LustreError: 103479:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.152@tcp: failed processing log, type 4: rc = -110 [ 3134.133777] Lustre: *** cfs_fail_loc=197, val=0*** [ 3134.137107] Lustre: Skipped 65 previous similar messages [ 3154.720207] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3154.732041] Lustre: Skipped 4 previous similar messages [ 3162.054833] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3170.627409] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3173.344873] Lustre: server umount lustre-OST0000 complete [ 3184.751873] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:49:58 (1787464198) [ 3208.121298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3213.870310] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3224.822112] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3229.774903] LustreError: 105131:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3229.804437] LustreError: 105131:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3229.815273] LustreError: 105131:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3230.668303] Lustre: server umount lustre-MDT0000 complete [ 3270.033542] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 3283.512129] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3330.467534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3330.811360] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3330.893597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3330.984631] Lustre: lustre-MDT0000: new disk, initializing [ 3331.275628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3336.945788] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3350.450676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3350.640752] Lustre: 109315:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3350.962781] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3358.693158] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3371.885941] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3378.540930] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3386.616026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3386.976679] Lustre: 111284:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3392.773924] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3396.373341] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3407.996852] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3413.695911] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3414.052744] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3420.595735] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3420.907992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3423.714187] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3423.720665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3423.727601] LustreError: Skipped 5 previous similar messages [ 3423.734547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3423.734556] Lustre: Skipped 13 previous similar messages [ 3423.768775] Lustre: Skipped 4 previous similar messages [ 3429.941347] Lustre: server umount lustre-OST0000 complete [ 3440.204432] Lustre: server umount lustre-MDT0000 complete [ 3444.208167] LustreError: 109326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3444.236511] LustreError: 109326:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 3444.937209] LustreError: 110110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787464461 with bad export cookie 1718502083734517165 [ 3444.950907] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3444.956676] LustreError: 110110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3445.799599] Lustre: server umount lustre-MDT0001 complete [ 3456.543231] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:54:30 (1787464470) [ 3468.578643] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3472.568508] Key type lgssc unregistered [ 3473.006440] LNet: 113777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3473.012057] LNetError: 113777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3473.034306] LNet: Removed LNI 192.168.201.152@tcp [ 3474.254328] Key type .llcrypt unregistered [ 3474.256193] Key type ._llcrypt unregistered [ 3499.598165] Key type ._llcrypt registered [ 3499.601197] Key type .llcrypt registered [ 3499.812529] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 3522.002835] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3524.002133] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3524.037371] alg: No test for adler32 (adler32-zlib) [ 3525.314195] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 3525.685710] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3527.485262] Key type lgssc registered [ 3529.496917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3586.499722] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3586.552572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3588.014719] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3588.154875] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3588.272735] Lustre: lustre-MDT0000: new disk, initializing [ 3588.443844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3588.475116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3594.953583] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3608.697726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3609.227419] Lustre: lustre-OST0000: new disk, initializing [ 3609.234503] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3609.259561] Lustre: Skipped 1 previous similar message [ 3609.266691] Lustre: 117956:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3609.400790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3617.213702] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3617.938347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3617.946302] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3617.976935] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3631.031200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3631.152178] Lustre: lustre-OST0001: new disk, initializing [ 3631.156205] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3631.161609] Lustre: 118821:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3631.231964] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3639.632055] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3640.468329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3640.497059] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3640.554838] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3650.531198] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3650.536790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3650.551942] Lustre: Skipped 1 previous similar message [ 3650.574429] Lustre: Skipped 1 previous similar message [ 3655.660249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3655.669507] Lustre: Skipped 1 previous similar message [ 3656.519924] Lustre: server umount lustre-MDT0000 complete [ 3665.625597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3671.239639] LustreError: 117143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787464688 with bad export cookie 10243611664343206755 [ 3671.249181] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3671.255395] LustreError: 117143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3671.358071] Lustre: server umount lustre-OST0000 complete [ 3676.520358] Lustre: server umount lustre-OST0001 complete [ 3688.969552] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:58:22 (1787464702) [ 3717.525570] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:58:51 (1787464731) [ 3729.436681] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3760.585784] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:59:34 (1787464774) [ 3811.872040] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 3824.646842] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3873.909158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3874.291856] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3874.362262] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3874.495366] Lustre: lustre-MDT0000: new disk, initializing [ 3874.634985] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3874.669626] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3879.973491] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3894.141583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3894.376292] Lustre: 126594:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3894.517255] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3894.522415] Lustre: Skipped 1 previous similar message [ 3894.646388] Lustre: lustre-MDT0001: new disk, initializing [ 3894.846214] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3894.966233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3895.010349] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3901.204087] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3914.656776] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3922.674446] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3932.962157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3933.425906] Lustre: lustre-OST0000: new disk, initializing [ 3933.431964] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3933.455099] Lustre: 128565:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3933.600550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3941.344635] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3943.018640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3943.034731] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3943.138109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3953.258882] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3961.358607] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3961.687181] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3970.238604] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3970.726332] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3974.125326] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3974.131887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3974.153687] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3979.238951] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3979.247561] Lustre: Skipped 2 previous similar messages [ 3979.746561] Lustre: server umount lustre-OST0000 complete [ 3986.924508] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3986.939137] LustreError: Skipped 1 previous similar message [ 3986.955189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3986.975255] Lustre: Skipped 1 previous similar message [ 3986.985631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3989.475318] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3991.887820] Lustre: server umount lustre-MDT0000 complete [ 3996.490748] LustreError: 126588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787465013 with bad export cookie 10243611664343207973 [ 3996.492375] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3996.516507] LustreError: 126588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3997.219725] Lustre: server umount lustre-MDT0001 complete [ 4009.262967] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:03:42 (1787465022) [ 4020.337405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4021.306874] LustreError: 130979:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4021.368332] LustreError: 130979:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4021.481504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4026.884790] LustreError: 130980:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4028.556942] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4031.993254] LustreError: 130979:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.115500] LustreError: 130980:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.145193] LustreError: 130980:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4042.782103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4043.243813] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4043.252137] Lustre: Skipped 2 previous similar messages [ 4049.604965] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4059.685509] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4067.683652] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4076.726475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4077.337713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4077.345426] Lustre: Skipped 1 previous similar message [ 4085.845662] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4095.392135] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4101.483774] Lustre: *** cfs_fail_loc=159, val=0*** [ 4118.187759] Lustre: lustre-MDT0000: Client 3dd62b1e-66d3-4087-b6e1-5b50c15acab1 (at 192.168.201.52@tcp) reconnecting [ 4120.725118] Lustre: *** cfs_fail_loc=159, val=0*** [ 4120.737388] Lustre: Skipped 6 previous similar messages [ 4137.158456] Lustre: lustre-MDT0000: Client 3dd62b1e-66d3-4087-b6e1-5b50c15acab1 (at 192.168.201.52@tcp) reconnecting [ 4155.387493] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4155.389913] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4155.390953] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4155.426184] Lustre: Skipped 1 previous similar message [ 4161.087479] Lustre: server umount lustre-OST0000 complete [ 4165.602669] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4171.333641] Lustre: server umount lustre-MDT0000 complete [ 4175.725713] LustreError: 130960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787465192 with bad export cookie 10243611664343208855 [ 4175.733508] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4175.736636] LustreError: 130960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4176.370591] Lustre: server umount lustre-MDT0001 complete [ 4196.048289] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 4202.685530] Key type lgssc unregistered [ 4203.246508] LNet: 134785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4203.267830] LNetError: 134785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4203.298700] LNet: Removed LNI 192.168.201.152@tcp [ 4204.870536] Key type .llcrypt unregistered [ 4204.872807] Key type ._llcrypt unregistered [ 4236.947023] Key type ._llcrypt registered [ 4236.948826] Key type .llcrypt registered [ 4237.089228] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 4239.081331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4239.111214] alg: No test for adler32 (adler32-zlib) [ 4240.365379] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 4240.733593] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 4242.416233] Key type lgssc registered [ 4243.953908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4253.486993] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4253.498514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4255.069757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4260.053971] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4260.330885] LustreError: 136202:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4260.345415] LustreError: 136202:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4264.421390] LustreError: 136203:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4269.544935] LustreError: 136202:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4270.869971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4271.609630] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4277.742803] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4288.751575] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4297.625390] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4307.061292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4307.490458] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4316.472947] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4324.928508] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4327.328264] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4338.866041] Lustre: *** cfs_fail_loc=159, val=0*** [ 4354.184138] Lustre: lustre-MDT0000: Client 317ddf98-a3e0-46bc-8a87-8755825fb487 (at 192.168.201.52@tcp) reconnecting [ 4356.926870] Lustre: *** cfs_fail_loc=159, val=0*** [ 4356.930726] Lustre: Skipped 2 previous similar messages [ 4373.160093] Lustre: lustre-MDT0001: Client 317ddf98-a3e0-46bc-8a87-8755825fb487 (at 192.168.201.52@tcp) reconnecting [ 4375.284673] Lustre: *** cfs_fail_loc=159, val=0*** [ 4391.547821] Lustre: lustre-MDT0001: Client 317ddf98-a3e0-46bc-8a87-8755825fb487 (at 192.168.201.52@tcp) reconnecting [ 4397.722989] Lustre: 139243:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4401.289279] Lustre: *** cfs_fail_loc=159, val=0*** [ 4401.313062] Lustre: Skipped 4 previous similar messages [ 4417.187119] Lustre: lustre-MDT0000: Client 99794f68-af07-4d7e-9924-83884b555cd8 (at 192.168.201.52@tcp) reconnecting [ 4419.499118] Lustre: *** cfs_fail_loc=159, val=0*** [ 4419.512245] Lustre: Skipped 16 previous similar messages [ 4436.193705] Lustre: lustre-MDT0000: Client 99794f68-af07-4d7e-9924-83884b555cd8 (at 192.168.201.52@tcp) reconnecting [ 4439.291589] Lustre: 139533:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4445.670836] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4445.680146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4445.694401] LustreError: Skipped 1 previous similar message [ 4445.728654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4447.743715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4447.762707] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4447.763808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4447.804419] LustreError: Skipped 1 previous similar message [ 4451.533660] Lustre: server umount lustre-OST0000 complete [ 4457.976724] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4457.996557] LustreError: 138962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4458.216185] Lustre: server umount lustre-MDT0000 complete [ 4463.097154] LustreError: 136197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4463.129023] LustreError: 136197:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4464.413498] LustreError: 136937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787465481 with bad export cookie 4549328446983899427 [ 4464.452759] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4465.082653] Lustre: server umount lustre-MDT0001 complete [ 4484.741516] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 4489.093260] Key type lgssc unregistered [ 4489.492226] LNet: 140593:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4489.501610] LNetError: 140593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4489.548900] LNet: Removed LNI 192.168.201.152@tcp [ 4491.254574] Key type .llcrypt unregistered [ 4491.257189] Key type ._llcrypt unregistered [ 4532.034816] Key type ._llcrypt registered [ 4532.036932] Key type .llcrypt registered [ 4532.131667] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 4533.815098] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4534.044495] alg: No test for adler32 (adler32-zlib) [ 4535.146355] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 4535.420628] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 4537.088210] Key type lgssc registered [ 4538.580597] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4547.732971] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4547.793111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4549.611180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4557.051148] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4559.842124] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4559.897334] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4564.966807] LustreError: 142011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4570.084278] LustreError: 142010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4571.693325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.310138] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4578.519360] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4589.019705] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4597.478096] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4605.532226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4605.720208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4612.891373] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4618.070977] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4620.126933] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4624.887388] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4632.816976] Lustre: 144595:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4637.664102] Lustre: 144644:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4639.538402] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4640.595490] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4645.322643] Lustre: 144792:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4646.614238] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4647.857840] Lustre: 144891:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4654.053364] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4654.058265] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4654.071437] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4656.100246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4656.110322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4656.130996] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4660.122855] Lustre: server umount lustre-OST0000 complete [ 4666.341291] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4666.377382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4666.380219] Lustre: Skipped 1 previous similar message [ 4671.465089] LustreError: 142006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4671.488321] LustreError: 142006:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4671.508445] Lustre: server umount lustre-MDT0000 complete [ 4676.580301] LustreError: 142007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4676.610865] LustreError: 142007:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4677.225881] LustreError: 141992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787465694 with bad export cookie 12035421750570363115 [ 4677.235336] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4677.241109] LustreError: 141992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4677.861143] Lustre: server umount lustre-MDT0001 complete [ 4692.187797] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 4695.607589] Key type lgssc unregistered [ 4695.959962] LNet: 145955:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4695.979311] LNetError: 145955:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4695.999359] LNet: Removed LNI 192.168.201.152@tcp [ 4697.104935] Key type .llcrypt unregistered [ 4697.113581] Key type ._llcrypt unregistered [ 4728.266375] Key type ._llcrypt registered [ 4728.271708] Key type .llcrypt registered [ 4728.441888] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 4729.767177] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4730.020830] alg: No test for adler32 (adler32-zlib) [ 4731.397954] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 4731.809208] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 4733.648181] Key type lgssc registered [ 4735.840733] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4747.071762] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4747.095715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4748.703200] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4753.655755] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4759.010401] LustreError: 147374:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4759.032635] LustreError: 147374:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4763.828591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4764.130364] LustreError: 147373:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4764.619729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4770.492033] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4779.966110] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4787.144490] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4795.696576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4796.241940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4804.343492] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4809.606102] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4813.772801] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4818.817521] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4824.623293] Lustre: *** cfs_fail_loc=159, val=0*** [ 4841.174251] Lustre: lustre-MDT0000: Client cddd5728-f8c3-48a3-88b7-ed149eb799d1 (at 192.168.201.52@tcp) reconnecting [ 4849.633896] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4849.641216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4849.647335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4849.654286] Lustre: Skipped 1 previous similar message [ 4849.669721] Lustre: Skipped 1 previous similar message [ 4854.770033] LustreError: 149202:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4854.902127] Lustre: server umount lustre-OST0000 complete [ 4859.876153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4859.884053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4859.886492] Lustre: Skipped 1 previous similar message [ 4859.927163] Lustre: Skipped 1 previous similar message [ 4864.994769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4865.012922] Lustre: Skipped 1 previous similar message [ 4866.232835] Lustre: server umount lustre-MDT0000 complete [ 4870.113365] LustreError: 149925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4870.137605] LustreError: 149925:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4872.108928] LustreError: 147354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787465888 with bad export cookie 14217381489725441966 [ 4872.113302] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4872.143859] LustreError: 147354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4872.712602] Lustre: server umount lustre-MDT0001 complete [ 4887.849713] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 4892.399858] Key type lgssc unregistered [ 4892.929083] LNet: 151044:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4892.955779] LNetError: 151044:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4892.982995] LNet: Removed LNI 192.168.201.152@tcp [ 4893.887482] Key type .llcrypt unregistered [ 4893.889167] Key type ._llcrypt unregistered [ 4925.085732] Key type ._llcrypt registered [ 4925.088677] Key type .llcrypt registered [ 4925.224604] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 4926.575997] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4926.687707] alg: No test for adler32 (adler32-zlib) [ 4927.899288] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 4928.161569] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 4929.848292] Key type lgssc registered [ 4931.167797] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4940.929388] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4940.971854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4942.858211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4949.046048] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4953.058928] LustreError: 152461:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4953.074342] LustreError: 152461:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4958.177633] LustreError: 152460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4960.264895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4960.972196] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4967.713452] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4977.843794] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4987.433441] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4998.858730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4999.276770] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5005.446322] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5008.432747] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5012.640139] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5020.041783] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5041.629487] Lustre: lustre-OST0000: evicting d44710a5-e11a-443a-994a-1bb54912407c (at 192.168.201.52@tcp) by administrative request [ 5041.641531] Lustre: Skipped 1 previous similar message [ 5072.356475] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5072.367910] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5072.402943] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5074.407346] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5074.413098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5074.434402] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5078.011862] Lustre: server umount lustre-OST0000 complete [ 5083.621000] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5083.637199] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.652059] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5083.661299] Lustre: Skipped 1 previous similar message [ 5089.408645] Lustre: server umount lustre-MDT0000 complete [ 5089.763113] LustreError: 152455:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5089.773564] LustreError: 152455:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5094.824763] LustreError: 152441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787466111 with bad export cookie 13626680926218899132 [ 5094.831710] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5094.842947] LustreError: 152441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5095.478716] Lustre: server umount lustre-MDT0001 complete [ 5115.153355] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5119.967076] Key type lgssc unregistered [ 5120.604779] LNet: 156551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5120.623047] LNetError: 156551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5121.707995] LNet: Removed LNI 192.168.201.152@tcp [ 5123.864829] Key type .llcrypt unregistered [ 5123.868726] Key type ._llcrypt unregistered [ 5230.870690] Key type ._llcrypt registered [ 5230.874775] Key type .llcrypt registered [ 5231.058578] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5232.974421] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5233.050374] alg: No test for adler32 (adler32-zlib) [ 5234.372065] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 5234.774721] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 5236.544201] Key type lgssc registered [ 5238.331847] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5248.256536] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5248.275814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5250.036744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5256.084537] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5259.237313] LustreError: 159047:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5259.262785] LustreError: 159047:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5264.368890] LustreError: 159048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5268.186555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5268.744892] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5275.117691] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5286.075876] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5294.185233] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5302.615160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5302.937892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5311.497825] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5313.236149] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5319.373560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5322.555734] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5333.988453] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5334.008882] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5334.020519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5335.021627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5335.024250] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5340.134850] LustreError: 160877:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5340.200332] LustreError: 160877:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5340.393477] Lustre: server umount lustre-OST0000 complete [ 5350.369616] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5350.403488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5350.416395] Lustre: Skipped 2 previous similar messages [ 5353.150636] Lustre: server umount lustre-MDT0000 complete [ 5355.491771] LustreError: 159778:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5355.506840] LustreError: 159778:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5358.136366] LustreError: 159027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787466375 with bad export cookie 11973551333849327467 [ 5358.164475] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5358.769140] Lustre: server umount lustre-MDT0001 complete [ 5376.565604] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5380.580087] Key type lgssc unregistered [ 5380.944245] LNet: 162597:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5380.962954] LNetError: 162597:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5380.982538] LNet: Removed LNI 192.168.201.152@tcp [ 5382.530397] Key type .llcrypt unregistered [ 5382.535096] Key type ._llcrypt unregistered [ 5418.830730] Key type ._llcrypt registered [ 5418.832518] Key type .llcrypt registered [ 5418.912349] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5420.136764] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5420.145360] alg: No test for adler32 (adler32-zlib) [ 5421.336552] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 5421.767794] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 5423.528910] Key type lgssc registered [ 5425.338930] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5447.106129] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:27:40 (1787466460) [ 5470.096831] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 5485.512327] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5533.560925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5533.607046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5535.043365] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5535.065060] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5535.174273] Lustre: lustre-MDT0000: new disk, initializing [ 5535.284698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5535.318109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5542.062576] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5557.141995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5557.404416] Lustre: 166880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5557.524385] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5557.527727] Lustre: Skipped 1 previous similar message [ 5557.581356] Lustre: lustre-MDT0001: new disk, initializing [ 5557.750940] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5557.827407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5557.831975] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5563.717452] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5576.501539] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5584.413562] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5592.575718] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5592.991946] Lustre: lustre-OST0000: new disk, initializing [ 5592.996558] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5593.012058] Lustre: 168852:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5593.126129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5600.871925] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5602.403533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5602.414531] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5602.536745] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5612.548579] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5627.817888] Lustre: server umount lustre-OST0000 complete [ 5634.533780] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5634.548808] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5634.591557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5638.627809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5638.638648] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5643.750363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5643.763231] Lustre: Skipped 2 previous similar messages [ 5644.768202] Lustre: 163326:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787466645/real 1787466645] req@000000001bd349f2 x1874294407730688/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787466661 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5644.848116] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5645.275732] Lustre: server umount lustre-MDT0000 complete [ 5650.016137] Lustre: 163327:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787466650/real 1787466650] req@000000007618a1d6 x1874294407732352/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787466666 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5650.402933] LustreError: 166891:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5650.422212] LustreError: 166891:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 5650.558110] LustreError: 166870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787466667 with bad export cookie 7983346434644489379 [ 5650.564539] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5650.567350] LustreError: 166870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5651.366931] Lustre: server umount lustre-MDT0001 complete [ 5664.661246] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5668.628116] Key type lgssc unregistered [ 5669.049592] LNet: 170761:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5669.059053] LNetError: 170761:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5669.097517] LNet: Removed LNI 192.168.201.152@tcp [ 5670.220532] Key type .llcrypt unregistered [ 5670.226556] Key type ._llcrypt unregistered [ 5706.362502] Key type ._llcrypt registered [ 5706.366495] Key type .llcrypt registered [ 5706.600884] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5708.698617] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5708.770057] alg: No test for adler32 (adler32-zlib) [ 5710.013576] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 5710.427460] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 5712.216219] Key type lgssc registered [ 5714.196200] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5724.787347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5724.823590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5726.401699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5732.952811] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5736.934163] LustreError: 172178:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5736.987072] LustreError: 172178:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5742.050787] LustreError: 172179:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5744.755966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5745.315960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5751.400988] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5760.633456] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5768.704784] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5777.210443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5777.832053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5786.226533] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5796.358979] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5796.652399] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5804.444355] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5804.499675] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5804.518040] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5804.531343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.585264] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5806.156497] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5806.182034] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5806.200220] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5806.205832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.218189] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5807.553493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5807.571676] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5807.583574] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5807.587118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.595861] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5808.753793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.764900] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5809.768150] Lustre: Skipped 1 previous similar message [ 5809.797345] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5809.802738] Lustre: Skipped 1 previous similar message [ 5809.804972] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5809.807787] Lustre: Skipped 1 previous similar message [ 5809.809638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.818961] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5809.824336] Lustre: Skipped 1 previous similar message [ 5810.488484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.915601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.184783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.487944] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5814.494927] Lustre: Skipped 3 previous similar messages [ 5814.539176] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5814.549250] Lustre: Skipped 3 previous similar messages [ 5814.557469] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5814.563231] Lustre: Skipped 3 previous similar messages [ 5814.565375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.576146] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5814.581068] Lustre: Skipped 3 previous similar messages [ 5815.700109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.681158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.459097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.138696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.747184] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5822.755189] Lustre: Skipped 4 previous similar messages [ 5822.790048] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5822.819207] Lustre: Skipped 4 previous similar messages [ 5822.829441] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5822.838792] Lustre: Skipped 4 previous similar messages [ 5822.845906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.862827] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5822.871915] Lustre: Skipped 4 previous similar messages [ 5824.824514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.722327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.604437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.214925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.781175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.094608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.416691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.121212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.749092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.950759] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5838.953661] Lustre: Skipped 9 previous similar messages [ 5838.968534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5838.973438] Lustre: Skipped 9 previous similar messages [ 5838.976588] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5838.981647] Lustre: Skipped 9 previous similar messages [ 5838.984788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.995583] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5839.002744] Lustre: Skipped 9 previous similar messages [ 5839.816026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.786822] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5840.799594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.800381] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5842.008073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.033131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.767661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.477942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.330832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.214994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.172868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.404180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.453944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.660522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.286511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.919772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.997545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.978074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.615071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.617268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.689262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.738791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.043188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.168535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.231654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.490713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.714469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.082913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.307362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.631288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.753565] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5871.763218] Lustre: Skipped 28 previous similar messages [ 5871.802023] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5871.812627] Lustre: Skipped 28 previous similar messages [ 5871.817138] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5871.828599] Lustre: Skipped 28 previous similar messages [ 5871.831331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.842512] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5871.852976] Lustre: Skipped 28 previous similar messages [ 5872.968370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.886287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.014772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.051203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.962356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.188035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.701686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.189911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.415201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.887129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.057691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.215754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.206075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.324084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.036373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.356408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.224361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.319781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.326317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.426079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.799538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.243465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.558452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.299666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.089596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.008648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.641552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.812722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.893845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.964920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.663977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.952819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.388741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.532782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.664308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.563294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.709630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.690565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.789921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.863360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.842063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.913836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.871649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.748584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.773321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.949331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.124582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.258813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.397113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.414355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.243402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.293538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.453251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.676591] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5936.685472] Lustre: Skipped 53 previous similar messages [ 5936.712930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5936.721489] Lustre: Skipped 53 previous similar messages [ 5936.725380] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5936.730684] Lustre: Skipped 53 previous similar messages [ 5936.735178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.761197] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5936.763659] Lustre: Skipped 53 previous similar messages [ 5937.718049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.948422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.194268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.162740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.148750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.006922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.095415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.141789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.262684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.312049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.299973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.369269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.417592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.483804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.446087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.660846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.890559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.190228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.253667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.794655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.579564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.468409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.800492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.338492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.971858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.922941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.625627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.413805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.537757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.648509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.076546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.725528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.086620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.618102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.765569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.916549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.149388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.366385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.688649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.915908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.042570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.316473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.651195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.011205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.434051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.710129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.278667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.409265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.586752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.707409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.030720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.333325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.423070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.842469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.045844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.105540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.226836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.398506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.812316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.356037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.534968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.854143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.924854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.964105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.979993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.237355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.329646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.636987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.938235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.073405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.179131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.320635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.791886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.217330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.487846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.012631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.507109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.809398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.999208] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6038.010758] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6038.030462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.591766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.808098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.038714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.846525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.903566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.023484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.033718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.119515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.292807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.593618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.842013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.874915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.101608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.051130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.455199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.665098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.952942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.448236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.842589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.931171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.239064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.456319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.063030] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6066.078530] Lustre: Skipped 101 previous similar messages [ 6066.109668] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6066.117158] Lustre: Skipped 101 previous similar messages [ 6066.121115] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6066.124278] Lustre: Skipped 101 previous similar messages [ 6066.128126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.137711] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6066.151664] Lustre: Skipped 101 previous similar messages [ 6067.273195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.400073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.536543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.951724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.326872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.682276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.041561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.324506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.259901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.543900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.818953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.281476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.670988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.093633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.508033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.780783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.120391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.332377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.703277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.923735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.924956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.296361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.331704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.331167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.430942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.731144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.152604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.519550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.961383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.324585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.519707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.829137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.157234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.453445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.743594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.643334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.012573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.487559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.063750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.390356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.923261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.342322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.755565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.369394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.793155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.561396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.117912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.289860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.584375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.964912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.945240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.582395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.671806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.203812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.573327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.825588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.911981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.687883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.303388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.738110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.207765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.816586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.406556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.171519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.008153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.419465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.999452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.621770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.273955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.143934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.819158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.341612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.800653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.115749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.274183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.480376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.784234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.188795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.711694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.851942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.063330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.322315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.621908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.822945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.381763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.427289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.802241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.009203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.043475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.334247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.325167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.584699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.690219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.030120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.405908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.383584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.604680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.791376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.965088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.292952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.432859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.563328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.865750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.245150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.371320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.718324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.741230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.314206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.816468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.330312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.624048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.960537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.215054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.166536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.652236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.862697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.183899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.340571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.749510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.892283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.213908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.623061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.788869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.985444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.280884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.523290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.817204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.075733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.047956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.296077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.603137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.738263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.756384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.971790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.308572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.741920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.254955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.653651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.024161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.195595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.279789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.406584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.763918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.136504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.464329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.689527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.828990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.065820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.268872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.546357] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6262.572124] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6262.579185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.704282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.812184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.134465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.405693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.678887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.959512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.411541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.949260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.291680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.408235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.691650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.900556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.900332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.334822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.530248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.000606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.412240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.706246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.124309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.532358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.402291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.566594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.682651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.880469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.289938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.637366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.069411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.132064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.389492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.691190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.000182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.399409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.718168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.034039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.407698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.555727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.898935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.448352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.900088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.303987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.277034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.544820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.122325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.095095] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6322.108523] Lustre: Skipped 193 previous similar messages [ 6322.148799] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6322.166040] Lustre: Skipped 193 previous similar messages [ 6322.178514] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6322.181723] Lustre: Skipped 193 previous similar messages [ 6322.199199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.215553] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6322.227811] Lustre: Skipped 193 previous similar messages [ 6323.893568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.409176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.544203] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6326.554575] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6326.556025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.314285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.512660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.058567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.349684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.043704] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6333.071675] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6333.133536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.135098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.785655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.424415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.898857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.287091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.787244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.439756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.664933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.796207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.349514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.871288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.097175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.307676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.743925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.166274] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6355.185442] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6355.197252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.420238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.681863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.302972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.606591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.041599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.471189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.671372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.886648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.184969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.511262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.864259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.600457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.984305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.492319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.819711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.911037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.236339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.741687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.092684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.453394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.621101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.349524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.670812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.881810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.011930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.172572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.401739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.894199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.295692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.605182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.912510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.935641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.198136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.195300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.473184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.929469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.238943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.602986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.781208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.954154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.907516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.262482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.367730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.514951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.746169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.485238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.952421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.581151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.610113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.355505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.848563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.081045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.246440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.772456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.589032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.000201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.163344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.387856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.506884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.964907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.005388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.344461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.493498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.804742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.340165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.431859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.965930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.836295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.841939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.230554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.511195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.499304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.400144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.304337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.452275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.375748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.566687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.753433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.040458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.050855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.958060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.807832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.268915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.200650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.378382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.528526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.098812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.316550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.455340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.671842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.074046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.651400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.094942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.270137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.520789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.730254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.998778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.932513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.467227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.891215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.436834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.836424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.135255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.448774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.900978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.247513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.726321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.950818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.294874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.540372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.864816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.322815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.467468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.763617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.128475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.084721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.694698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.005893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.384146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.558355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.888921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.537122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.879152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.215750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.308364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.343462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.371248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.537550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.814795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.934521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.159077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.280991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.418862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.639815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.042175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.351529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.754544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.070246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.517346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.664167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.381523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.778363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.162527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.814056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.354641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.843902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.269436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.545667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.801329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.969845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.225263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.368067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.669366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.787321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.198901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.460292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.004132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.409703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.657645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.035684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.346797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.686536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.816250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.130589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.191849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.469489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.895239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.107864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.281238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.520298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.651816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.006865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.198133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.634616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.993164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.290950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.646947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.298943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.327676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.644738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.938556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.250645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.737812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.242251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.518660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.805399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.002632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.378417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.707684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.109186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.564462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.830816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.068426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.298100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.636657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.763665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.138286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.189278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.605120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.631330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.798717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.092856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.332623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.663543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.814779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.914246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.160615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.284221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.803777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.117309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.390353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.664276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.013823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.240737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.586672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.666984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.192096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.621848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.258053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.764882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.279995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.770563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.829861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.562666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.678824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.077317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.268347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.683941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.381967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.625380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.760432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.367197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.494984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.950146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.405050] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6658.421981] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6658.438370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.927943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.366175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.347851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.513569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.019659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.551118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.990506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.084934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.247621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.486780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.501551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.431526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.643956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.902508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.307824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.593385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.649918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.811083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.346511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.769528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.079347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.390951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.674083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.270595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.626515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.955564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.133180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.244945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.733693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.953093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.169319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.292385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.667134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.058524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.246885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.306225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.503088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.787194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.189403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.540342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.627720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.715847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.976990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.304485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.505560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.224015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.398130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.596262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.881697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.476885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.244497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.809476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.607741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.654666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.914615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.119677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.274387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.355789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.182450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.746231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.728861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.216517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.815801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.787337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.378905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.015477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.894900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.382496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.603877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.123301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.021753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.997789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.923412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.195507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.562943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.979087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.707989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.544851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.605771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.806232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.090423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.363260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.837309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.281644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.560191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.827937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.165967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.164050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.006439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.360830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.001546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.623559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.296980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.143792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.532393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.770716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.207339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.818297] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6803.845528] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6803.882613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.584997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.717724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.494551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.625535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.283396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.474991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.668786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.871879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.790872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.200469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.130343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.654705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.281759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.608044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.830759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.075578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.412982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.624760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.838337] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6834.848077] Lustre: Skipped 374 previous similar messages [ 6834.872433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6834.886233] Lustre: Skipped 374 previous similar messages [ 6834.894800] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6834.900940] Lustre: Skipped 374 previous similar messages [ 6834.906455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.917375] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6834.936397] Lustre: Skipped 374 previous similar messages [ 6836.411831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.697728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.927584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.334634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.355312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.532695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.919765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.160775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.606116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.864539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.224667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.160639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.351284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.548650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.093156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.415571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.215509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.568913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.685261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.963438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.112381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.436853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.763824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.049078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.113674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.340686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.832395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.245145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.259684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.514233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.955622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.316507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.725584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.064718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.187747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.674595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.826510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.040844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.259122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.832745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.050949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.563990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.806237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.872914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.046750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.099619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.292738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.418607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.450404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.624649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.760090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.709457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.753023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.779034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.058330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.056053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.216826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.311516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.348728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.350683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.623057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.576721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.942478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.958025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.376974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6916.473248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.702039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.869912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.078607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.455313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6923.787758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.582846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.807955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.145510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.590163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6931.692169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.630592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.484688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.403846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.331871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6938.479763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.585875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.830946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6941.894431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6942.795214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6943.980427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.921865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6946.058958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6947.189385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.331449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.511442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.367345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.416831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.362206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6953.541122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.553779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.607692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6956.525541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.595934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.592219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6959.626879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.801427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6961.992845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6963.176703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.287253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.191651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.577930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6967.507574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.845402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6970.185544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6971.435109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.576562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6973.693903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.868507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.936627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6976.866909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.410346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.895738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6981.142901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.043241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.301860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6984.584569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.885336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.008698] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6988.030830] LustreError: 172198:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 6988.040949] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6988.054297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.056744] LustreError: 172198:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 6989.262121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.670793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.165266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.452848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6994.720906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.484635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.200580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6999.654927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.574299] Lustre: server umount lustre-OST0000 complete [ 7188.961414] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7188.976577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7193.064993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7193.082128] Lustre: Skipped 1 previous similar message [ 7198.176445] Lustre: 171492:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787468199/real 1787468199] req@00000000078273c1 x1874294713344512/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787468215 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7198.178542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7198.225483] Lustre: Skipped 1 previous similar message [ 7198.351952] Lustre: server umount lustre-MDT0000 complete [ 7199.203753] Lustre: 171490:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787468200/real 1787468200] req@0000000058db77d2 x1874294713344768/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787468216 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7202.792619] LustreError: 172160:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787468219 with bad export cookie 16772498237828607733 [ 7202.803263] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7203.500716] Lustre: server umount lustre-MDT0001 complete [ 7220.912767] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 7223.938234] Key type lgssc unregistered [ 7224.253778] LNet: 222724:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7224.258830] LNetError: 222724:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7224.273384] LNet: Removed LNI 192.168.201.152@tcp [ 7225.229098] Key type .llcrypt unregistered [ 7225.232108] Key type ._llcrypt unregistered [ 7255.114665] Key type ._llcrypt registered [ 7255.118595] Key type .llcrypt registered [ 7255.296071] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 7256.985898] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7257.010705] alg: No test for adler32 (adler32-zlib) [ 7258.345270] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 7258.804022] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 7260.568214] Key type lgssc registered [ 7261.863817] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7271.618632] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7271.668229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7273.396862] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7278.585640] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7283.684199] LustreError: 224140:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7283.697992] LustreError: 224140:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7288.704572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7288.802300] LustreError: 224141:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7289.148618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7294.895718] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7303.488677] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7309.764371] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7318.928027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7319.272115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7328.126565] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7339.424628] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7343.875670] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:13971 to 0x280000401:14017) [ 7351.265474] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7353.393884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7353.416179] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7353.436126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7353.460188] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7353.468086] LustreError: 224160:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7357.565855] Lustre: lustre-OST0000: Not available for connect from 192.168.201.52@tcp (stopping) [ 7359.460316] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7359.467927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7359.490993] Lustre: Skipped 1 previous similar message [ 7361.514546] Lustre: server umount lustre-OST0000 complete [ 7373.562896] LustreError: 224160:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7397.399272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7397.820103] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7399.863355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.873918] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.874447] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7399.896480] Lustre: Skipped 1 previous similar message [ 7407.679895] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7418.360088] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7425.728877] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 5 sec [ 7432.022575] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff899744df0800.ost_server_uuid 50 [ 7433.966547] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff899744df0800.ost_server_uuid in FULL state after 0 sec [ 7439.337269] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7439.351926] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7439.381050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7439.391128] Lustre: Skipped 1 previous similar message [ 7440.869473] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7441.213361] Lustre: server umount lustre-OST0000 complete [ 7448.033317] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7448.040871] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7448.052803] Lustre: Skipped 1 previous similar message [ 7448.056987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7448.066322] Lustre: Skipped 2 previous similar messages [ 7456.229610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7456.244050] Lustre: Skipped 3 previous similar messages [ 7461.346830] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7461.992793] Lustre: server umount lustre-MDT0000 complete [ 7465.441804] LustreError: 224140:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7465.470340] LustreError: 224140:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7467.289806] LustreError: 224122:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787468484 with bad export cookie 13808605604350583613 [ 7467.301781] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7467.608538] Lustre: server umount lustre-MDT0001 complete [ 7482.890910] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 7486.268433] Key type lgssc unregistered [ 7486.552404] LNet: 229399:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7486.558422] LNetError: 229399:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7486.572621] LNet: Removed LNI 192.168.201.152@tcp [ 7487.804431] Key type .llcrypt unregistered [ 7487.806956] Key type ._llcrypt unregistered [ 7525.484764] Key type ._llcrypt registered [ 7525.487490] Key type .llcrypt registered [ 7525.609424] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 7527.000964] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7527.048873] alg: No test for adler32 (adler32-zlib) [ 7528.136951] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 7528.382679] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 7530.040251] Key type lgssc registered [ 7531.501732] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7542.158329] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7542.185175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7543.568781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7549.172567] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7554.022745] LustreError: 231132:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7554.051729] LustreError: 231132:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7558.130473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7558.566470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7564.075227] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7567.382354] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7575.606104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7576.102616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7584.767609] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7588.416242] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14019 to 0x280000401:14049) [ 7597.072761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7597.207416] Lustre: lustre-OST0001: new disk, initializing [ 7597.209727] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7597.214158] Lustre: 233440:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7597.270335] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7604.837351] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7606.946329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7606.971273] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7607.155248] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7617.193753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7625.318086] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7640.553874] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7640.567060] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7640.577257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7643.106972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7643.108334] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7643.140522] Lustre: Skipped 2 previous similar messages [ 7644.606956] Lustre: server umount lustre-MDT0000 complete [ 7648.221520] LustreError: 232639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787468665 with bad export cookie 11507921258736254672 [ 7648.248362] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7648.251827] LustreError: 232639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7648.268168] LustreError: 232081:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7648.275742] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7648.278452] LustreError: 232081:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7648.295703] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7648.300978] Lustre: Skipped 4 previous similar messages [ 7648.557239] Lustre: server umount lustre-MDT0001 complete [ 7654.498712] Lustre: server umount lustre-OST0000 complete [ 7659.177152] Lustre: server umount lustre-OST0001 complete [ 7682.860723] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 7698.468053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7699.038610] LustreError: 236549:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7699.109947] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7704.559512] LustreError: 236550:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7708.121602] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7708.643033] LustreError: 236550:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7718.887023] LustreError: 236549:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7718.897856] LustreError: 236549:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7719.334406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7719.663422] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7724.179873] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7727.448533] Lustre: 237693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7736.109348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7736.622676] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7743.804325] LustreError: 238046:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7743.840103] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14052 to 0x280000401:14081) [ 7744.217321] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7754.218506] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7762.073947] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7770.708422] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7774.956015] Lustre: 239563:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7790.591667] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7790.611335] Lustre: Skipped 3 previous similar messages [ 7790.617955] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7795.695852] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7795.714803] Lustre: Skipped 3 previous similar messages [ 7795.884909] Lustre: server umount mylustre-MDT0000 complete [ 7799.847116] LustreError: 236527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787468816 with bad export cookie 11507921258736257444 [ 7799.867652] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7799.871049] LustreError: 236527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7800.343320] Lustre: server umount mylustre-MDT0001 complete [ 7814.849584] Lustre: server umount mylustre-OST0000 complete [ 7816.996269] Lustre: 230396:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787468817/real 1787468817] req@ffff89dfd2b75880 x1874296617199488/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787468833 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7817.030505] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7817.047621] Lustre: Skipped 1 previous similar message [ 7819.031324] Lustre: server umount mylustre-OST0001 complete [ 7849.309779] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 7862.932513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7863.813614] LustreError: 241873:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7863.841675] LustreError: 241873:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 7863.921480] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7863.931393] Lustre: Skipped 1 previous similar message [ 7869.604364] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7881.817036] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7888.503477] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7893.715836] Lustre: 243019:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7904.651124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7909.143259] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14083 to 0x280000401:14113) [ 7913.660644] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7924.260945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7931.297958] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7940.950780] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7945.974529] Lustre: 244895:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7953.913915] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7953.937356] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7953.953757] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7953.956472] Lustre: Skipped 1 previous similar message [ 7956.537828] Lustre: server umount tfs-MDT0000 complete [ 7960.565426] LustreError: 241874:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7960.587306] LustreError: 241874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 7960.745638] LustreError: 242620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787468977 with bad export cookie 11507921258736259110 [ 7960.771336] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7961.065321] Lustre: server umount tfs-MDT0001 complete [ 7975.279974] Lustre: server umount tfs-OST0000 complete [ 7989.457256] Lustre: server umount tfs-OST0001 complete [ 8014.457398] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8028.357261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8028.967234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8028.973033] Lustre: Skipped 3 previous similar messages [ 8033.671906] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8043.952224] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8049.192441] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8052.227531] Lustre: 248286:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8059.561459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8067.448401] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8073.365390] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14083 to 0x280000401:14145) [ 8078.741869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8078.833301] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 8078.837996] Lustre: Skipped 5 previous similar messages [ 8086.705669] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8094.876697] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8099.131784] Lustre: 250155:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8106.756839] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:12:00 (1787469120) [ 8124.898995] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8124.905225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8124.939878] Lustre: Skipped 6 previous similar messages [ 8124.943345] Lustre: Skipped 2 previous similar messages [ 8130.819067] Lustre: server umount lustre-MDT0000 complete [ 8135.152746] LustreError: 247145:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8135.202585] LustreError: 247145:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8136.189259] LustreError: 247121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787469153 with bad export cookie 11507921258736260552 [ 8136.208724] LustreError: 247121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8136.212598] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8143.018582] Lustre: server umount lustre-MDT0001 complete [ 8162.275121] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8162.452931] Lustre: server umount lustre-OST0000 complete [ 8173.285657] Lustre: server umount lustre-OST0001 complete [ 8180.838726] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 8194.649763] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8246.460800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8247.015921] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8247.058243] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8247.271680] Lustre: lustre-MDT0000: new disk, initializing [ 8247.448994] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8247.467693] Lustre: Skipped 3 previous similar messages [ 8247.519407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8254.709521] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8267.866954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8268.132374] Lustre: 253366:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8268.298666] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8268.310719] Lustre: Skipped 1 previous similar message [ 8268.440723] Lustre: lustre-MDT0001: new disk, initializing [ 8268.621603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8268.633681] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8272.973248] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8283.654128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8284.101138] Lustre: lustre-OST0000: new disk, initializing [ 8284.109324] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8284.127467] Lustre: 254947:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8285.447869] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8285.466931] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8285.606715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8293.663814] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8310.810226] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8311.149274] Lustre: lustre-OST0001: new disk, initializing [ 8311.159840] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8311.175943] Lustre: 255820:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8312.485213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8312.495052] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8312.630684] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8319.080385] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8336.138915] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:15:49 (1787469349) [ 8358.004757] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:16:11 (1787469371) [ 8377.827031] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8377.827757] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8377.840036] LustreError: Skipped 1 previous similar message [ 8377.842371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8377.842388] Lustre: Skipped 9 previous similar messages [ 8377.886422] Lustre: Skipped 3 previous similar messages [ 8381.821522] Lustre: server umount lustre-OST0000 complete [ 8394.729659] LustreError: 254933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8394.802132] LustreError: 254933:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 8401.376161] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8402.027721] Lustre: server umount lustre-MDT0000 complete [ 8406.602677] LustreError: 253358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787469423 with bad export cookie 11507921258736261882 [ 8406.606043] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8406.613194] LustreError: 253358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8413.232758] Lustre: server umount lustre-MDT0001 complete [ 8429.489595] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 8505.304670] Lustre: server umount lustre-OST0001 complete [ 8513.886192] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 8533.369452] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8579.450609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8579.709706] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8579.733218] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8579.794438] Lustre: lustre-MDT0000: new disk, initializing [ 8579.928223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8579.939657] Lustre: Skipped 3 previous similar messages [ 8579.963504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8584.789435] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8598.295930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8598.553938] Lustre: 260684:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8598.631181] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8598.636037] Lustre: Skipped 1 previous similar message [ 8598.722722] Lustre: lustre-MDT0001: new disk, initializing [ 8598.991739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8599.021957] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8606.623801] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8617.984610] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8625.202630] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8634.024187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8634.519230] Lustre: lustre-OST0000: new disk, initializing [ 8634.523661] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8634.531171] Lustre: 262655:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8636.474573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8636.497073] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8636.661344] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8643.972982] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8656.282531] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8667.617211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8667.623800] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8667.628151] LustreError: Skipped 1 previous similar message [ 8667.636949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8667.636956] Lustre: Skipped 14 previous similar messages [ 8667.665647] Lustre: Skipped 5 previous similar messages [ 8671.368994] Lustre: server umount lustre-OST0000 complete [ 8675.810242] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8690.144919] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8690.425685] Lustre: server umount lustre-MDT0000 complete [ 8695.267159] LustreError: 260675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787469712 with bad export cookie 11507921258736263352 [ 8695.270141] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8695.284396] LustreError: 260675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8695.946471] Lustre: server umount lustre-MDT0001 complete [ 8711.469425] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 8715.612654] Key type lgssc unregistered [ 8716.056968] LNet: 264420:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8716.071832] LNetError: 264420:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8716.093130] LNet: Removed LNI 192.168.201.152@tcp [ 8717.448162] Key type .llcrypt unregistered [ 8717.450242] Key type ._llcrypt unregistered [ 8753.417081] Key type ._llcrypt registered [ 8753.423355] Key type .llcrypt registered [ 8753.628473] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8755.050496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8755.148088] alg: No test for adler32 (adler32-zlib) [ 8756.324552] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 8756.522944] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 8758.184196] Key type lgssc registered [ 8759.869711] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8769.927445] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8769.949871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8771.566769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8777.613918] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8781.797302] LustreError: 265893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8781.841972] LustreError: 265893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8786.920557] LustreError: 265892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8789.316605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8789.890910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8795.980983] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8806.214452] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8813.582296] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8823.959968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8824.607641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8833.427842] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8842.955741] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8843.072676] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8844.083766] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8848.355417] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8848.362621] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8848.393247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8848.399238] Lustre: Skipped 1 previous similar message [ 8852.736226] Lustre: server umount lustre-OST0000 complete [ 8858.615193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8858.617892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8858.636238] Lustre: Skipped 2 previous similar messages [ 8858.647292] Lustre: Skipped 1 previous similar message [ 8863.721602] LustreError: 265888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8863.750318] LustreError: 265888:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8864.091753] Lustre: server umount lustre-MDT0000 complete [ 8868.824577] LustreError: 265873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787469885 with bad export cookie 2723569402411603997 [ 8868.835835] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8868.842716] LustreError: 265873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8869.533429] Lustre: server umount lustre-MDT0001 complete [ 8883.852835] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 8886.938572] Key type lgssc unregistered [ 8887.238620] LNet: 269484:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8887.247664] LNetError: 269484:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8887.269843] LNet: Removed LNI 192.168.201.152@tcp [ 8888.082858] Key type .llcrypt unregistered [ 8888.085130] Key type ._llcrypt unregistered [ 8908.976294] Key type ._llcrypt registered [ 8908.978479] Key type .llcrypt registered [ 8909.102939] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8911.140940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8911.181382] alg: No test for adler32 (adler32-zlib) [ 8912.480241] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 8912.907232] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 8914.625051] Key type lgssc registered [ 8916.244431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8927.882721] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8927.933729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8943.521053] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8948.645887] LustreError: 270678:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.152@tcp: failed processing log, type 4: rc = -110 [ 8975.393187] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8982.731533] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8992.247647] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8994.959247] Lustre: server umount lustre-OST0000 complete [ 9015.430033] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 9019.193801] Key type lgssc unregistered [ 9019.584008] LNet: 272312:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9019.589961] LNetError: 272312:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9019.621709] LNet: Removed LNI 192.168.201.152@tcp [ 9020.687491] Key type .llcrypt unregistered [ 9020.693265] Key type ._llcrypt unregistered [ 9070.423311] Key type ._llcrypt registered [ 9070.424686] Key type .llcrypt registered [ 9070.539872] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 9071.804341] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9071.878596] alg: No test for adler32 (adler32-zlib) [ 9073.002866] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 9073.313445] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 9074.992783] Key type lgssc registered [ 9076.537248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9091.275980] loop0: detected capacity change from 0 to 204800000 [ 9091.519805] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9091.540890] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9092.019649] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9093.236797] loop0: detected capacity change from 0 to 204800000 [ 9093.500580] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9093.518054] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9093.936156] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9095.030211] loop0: detected capacity change from 0 to 204800000 [ 9095.225747] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9095.241158] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9095.359460] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9096.647539] loop0: detected capacity change from 0 to 204800000 [ 9096.868469] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9096.890571] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9097.049469] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9098.118825] loop0: detected capacity change from 0 to 204800000 [ 9098.138461] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9102.193106] loop0: detected capacity change from 0 to 204800000 [ 9102.229120] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9105.980958] loop0: detected capacity change from 0 to 204800000 [ 9106.035849] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9110.346802] loop0: detected capacity change from 0 to 204800000 [ 9110.413107] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9116.890890] loop0: detected capacity change from 0 to 204800000 [ 9117.143912] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9117.234831] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9117.437674] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9117.457135] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9117.484189] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9117.510727] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9117.550559] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9117.577101] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9117.609134] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9117.624878] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9117.636607] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9117.647131] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9117.657233] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9117.663960] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9117.676985] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9117.693728] Lustre: 275734:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9126.081826] Lustre: server umount MGS complete [ 9127.050498] loop0: detected capacity change from 0 to 204800000 [ 9127.151603] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9128.002737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9128.079364] Lustre: lustre-MDT0000: Aborting client recovery [ 9129.401469] loop1: detected capacity change from 0 to 204800000 [ 9129.631041] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9129.780658] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9129.798964] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9129.815335] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9129.837344] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9129.857846] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9129.876814] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9129.895441] Lustre: 276167:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9129.936890] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9130.259712] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9130.326772] Lustre: lustre-MDT0001: Aborting client recovery [ 9131.294078] loop2: detected capacity change from 0 to 204800000 [ 9131.526439] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9131.612331] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9131.632580] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9131.658606] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9131.682945] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9131.705730] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9131.724570] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9131.739820] Lustre: 276247:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9131.928746] Lustre: Found index 0 for lustre-OST0000, updating log [ 9131.968721] Lustre: 276264:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9132.043459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9132.066394] Lustre: lustre-OST0000: Aborting recovery [ 9133.088924] loop3: detected capacity change from 0 to 204800000 [ 9133.265516] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9133.348445] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9133.365955] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9133.380041] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9133.388773] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9133.397148] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9133.414693] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9133.423929] Lustre: 276329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9133.442253] Lustre: Found index 1 for lustre-OST0001, updating log [ 9133.460554] Lustre: 276334:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9133.535204] Lustre: lustre-OST0001: Aborting recovery [ 9134.744158] Lustre: lustre-MDT0000: trigger OI scrub by RPC for [0x200000402:0x7:0x0]/0 with flags 0x4a: rc = 0 [ 9137.689355] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9137.692991] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9137.713064] LustreError: 276111:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9137.731634] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9137.752464] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9140.736619] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9140.747229] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9142.822286] LustreError: 276078:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 543us: rc = -2 [ 9143.349726] LustreError: 276075:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 11837us: rc = -2 [ 9143.398247] LustreError: 276075:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 18 previous similar messages [ 9161.184765] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9161.188062] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9161.196050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9161.225049] Lustre: Skipped 3 previous similar messages [ 9162.833129] Lustre: server umount lustre-MDT0000 complete [ 9163.814820] LustreError: 276061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787470180 with bad export cookie 11530704124514601033 [ 9163.828787] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9163.836851] LustreError: 276061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9166.306294] LustreError: 276076:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9166.306733] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9166.320348] LustreError: 276076:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9166.342080] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9166.347683] Lustre: Skipped 4 previous similar messages [ 9168.353524] LustreError: 276589:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9168.354609] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9168.388054] Lustre: Skipped 1 previous similar message [ 9170.410850] Lustre: server umount lustre-MDT0001 complete [ 9186.272222] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9186.570980] Lustre: server umount lustre-OST0000 complete [ 9202.144173] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9202.416868] Lustre: server umount lustre-OST0001 complete [ 9212.544756] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:30:26 (1787470226) [ 9244.295616] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 9258.756067] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 9305.031684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9305.342439] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9305.401521] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9305.513622] Lustre: lustre-MDT0000: new disk, initializing [ 9305.698621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9305.711709] Lustre: Skipped 1 previous similar message [ 9305.734350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9311.537382] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9324.957134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9325.121229] Lustre: 280401:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9325.211518] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9325.217028] Lustre: Skipped 1 previous similar message [ 9325.346308] Lustre: lustre-MDT0001: new disk, initializing [ 9325.496515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9325.549107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9325.570262] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9330.877520] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9343.006877] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9349.626907] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9357.013205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9357.243263] Lustre: lustre-OST0000: new disk, initializing [ 9357.251159] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9357.260886] Lustre: 282371:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9357.364801] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9363.733818] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9367.650573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9367.666991] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9367.824592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9373.870337] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9378.985903] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9390.234700] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9394.458318] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9401.275451] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9444.841738] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9444.843128] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9444.853253] LustreError: Skipped 1 previous similar message [ 9444.853448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9444.894509] Lustre: Skipped 2 previous similar messages [ 9446.850211] Lustre: server umount lustre-OST0000 complete [ 9453.539318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9453.563259] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9453.585586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9453.590381] Lustre: Skipped 1 previous similar message [ 9457.087574] Lustre: server umount lustre-MDT0000 complete [ 9460.203120] LustreError: 280410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9460.239619] LustreError: 280410:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9462.355263] LustreError: 280395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787470479 with bad export cookie 11530704124514603749 [ 9462.371306] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9462.385774] LustreError: 280395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9463.218847] Lustre: server umount lustre-MDT0001 complete [ 9475.213333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9480.047307] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9485.849766] Lustre: server umount MGS complete [ 9500.634306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9501.252591] LustreError: 286197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9501.278756] LustreError: 286210:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9501.348322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9506.798183] LustreError: 286198:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9507.397266] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9517.030715] LustreError: 286198:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9517.074426] LustreError: 286198:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9519.073266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9525.080488] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9533.571074] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9539.645608] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9547.102211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9547.417708] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9547.429444] Lustre: Skipped 1 previous similar message [ 9555.942865] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9567.971359] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9606.115495] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9606.127151] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9606.148532] Lustre: Skipped 1 previous similar message [ 9606.158043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9606.163653] Lustre: Skipped 2 previous similar messages [ 9612.097424] Lustre: server umount lustre-OST0000 complete [ 9616.866115] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9616.874616] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9616.884261] Lustre: Skipped 1 previous similar message [ 9623.525822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9623.541139] Lustre: Skipped 6 previous similar messages [ 9631.200303] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9631.512214] Lustre: server umount lustre-MDT0000 complete [ 9633.761687] LustreError: 286197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9633.785811] LustreError: 286197:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9636.008942] LustreError: 288925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787470652 with bad export cookie 11530704124514605583 [ 9636.022815] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9636.027799] LustreError: 288925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9636.554784] Lustre: server umount lustre-MDT0001 complete [ 9650.913950] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 9654.685722] Key type lgssc unregistered [ 9655.008653] LNet: 290149:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9655.038840] LNetError: 290149:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9655.062080] LNet: Removed LNI 192.168.201.152@tcp [ 9656.287477] Key type .llcrypt unregistered [ 9656.293734] Key type ._llcrypt unregistered [ 9702.358247] Key type ._llcrypt registered [ 9702.360607] Key type .llcrypt registered [ 9702.479991] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 9719.298922] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 9720.714406] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9720.887182] alg: No test for adler32 (adler32-zlib) [ 9722.265402] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [ 9722.556921] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 9724.320184] Key type lgssc registered [ 9726.204895] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9769.863084] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9769.938659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9771.313402] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9771.361714] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9771.489433] Lustre: lustre-MDT0000: new disk, initializing [ 9771.641218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9771.683289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9776.757893] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9789.531924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9789.702198] Lustre: 294115:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9789.757383] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9789.762187] Lustre: Skipped 1 previous similar message [ 9789.837054] Lustre: lustre-MDT0001: new disk, initializing [ 9789.925159] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9789.964097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9789.976154] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9795.790279] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9807.299846] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9813.644475] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9821.433812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9821.793776] Lustre: lustre-OST0000: new disk, initializing [ 9821.800188] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9821.806871] Lustre: 296080:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9821.893482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9828.429793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9828.442972] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9828.563096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9829.294795] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9841.874399] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9850.301908] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9863.271378] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9868.861369] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9875.216561] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9920.997176] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9921.002481] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9921.015182] Lustre: Skipped 1 previous similar message [ 9921.040393] Lustre: Skipped 1 previous similar message [ 9926.114261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9926.126665] Lustre: Skipped 1 previous similar message [ 9926.744784] Lustre: server umount lustre-OST0000 complete [ 9933.282315] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9933.291933] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9933.303677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9936.363865] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9936.372601] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9938.150303] Lustre: server umount lustre-MDT0000 complete [ 9942.092727] LustreError: 294105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787470958 with bad export cookie 4831462015520633259 [ 9942.094838] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9942.112595] LustreError: 294105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9942.672241] Lustre: server umount lustre-MDT0001 complete [ 9952.407636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9958.244237] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9963.300361] Lustre: server umount MGS complete [ 9975.367448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9975.722652] LustreError: 299902:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9975.739888] LustreError: 299902:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9975.754406] LustreError: 299915:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9975.807879] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9980.905507] LustreError: 299903:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9982.023415] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9986.032858] LustreError: 299902:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9991.148695] LustreError: 299903:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9992.436881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9992.916389] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9999.279032] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10006.927172] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10013.456379] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10021.705799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10022.216920] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10030.082671] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10038.973928] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10081.763157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10081.775487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10081.797147] Lustre: Skipped 1 previous similar message [10081.811392] Lustre: Skipped 1 previous similar message [10087.346203] Lustre: server umount lustre-OST0000 complete [10092.001694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10092.012297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10092.021686] Lustre: Skipped 4 previous similar messages [10097.857086] Lustre: server umount lustre-MDT0000 complete [10102.032727] LustreError: 299883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787471118 with bad export cookie 4831462015520635086 [10102.037852] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10102.045273] LustreError: 299883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10102.424654] Lustre: server umount lustre-MDT0001 complete [10114.662336] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [10117.692595] Key type lgssc unregistered [10118.102527] LNet: 303879:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10118.109579] LNetError: 303879:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10118.127059] LNet: Removed LNI 192.168.201.152@tcp [10119.045479] Key type .llcrypt unregistered [10119.047298] Key type ._llcrypt unregistered [10143.322945] Key type ._llcrypt registered [10143.327368] Key type .llcrypt registered [10143.518484] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [10156.772831] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [10158.206909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10158.235904] alg: No test for adler32 (adler32-zlib) [10159.569938] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [10159.951830] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [10161.680343] Key type lgssc registered [10163.100792] Lustre: Echo OBD driver; http://www.lustre.org/ [10191.187152] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [10209.284381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10209.316107] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10210.594882] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10210.620198] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10210.699257] Lustre: lustre-MDT0000: new disk, initializing [10210.774457] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10210.795776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10216.625715] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10229.327928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10229.660843] Lustre: lustre-OST0000: new disk, initializing [10229.669229] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10229.676835] Lustre: Skipped 1 previous similar message [10229.690696] Lustre: 307908:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10229.845264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10236.541449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10236.557462] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10236.610451] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10238.586821] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11111.905887] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11111.914586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11113.752657] Lustre: server umount lustre-MDT0000 complete [11118.229865] LustreError: 308790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787472135 with bad export cookie 8943804976796107014 [11118.235541] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11118.242103] LustreError: 308790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11118.430463] Lustre: server umount lustre-OST0000 complete [11145.377500] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [11157.407456] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [11202.686712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11202.971696] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11203.054473] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11203.156683] Lustre: lustre-MDT0000: new disk, initializing [11203.228568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11203.277340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11208.917776] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11221.428926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11221.564736] Lustre: 312264:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11221.632554] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11221.640726] Lustre: Skipped 1 previous similar message [11221.811554] Lustre: lustre-MDT0001: new disk, initializing [11221.957352] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11221.995877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11221.999493] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11227.250734] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11237.973061] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11243.840874] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11250.511659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11250.815706] Lustre: lustre-OST0000: new disk, initializing [11250.821485] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11250.830899] Lustre: 314230:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11250.968507] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11256.968553] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11258.443126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11258.452542] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11258.597210] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11267.445432] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11272.881498] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11273.137884] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11277.759894] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11277.963441] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11284.451268] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11284.456488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11284.461641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11284.465217] LustreError: Skipped 1 previous similar message [11284.487805] Lustre: Skipped 1 previous similar message [11289.570801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11289.578057] Lustre: Skipped 1 previous similar message [11294.176903] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11294.464837] Lustre: server umount lustre-OST0000 complete [11298.790152] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11298.797326] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11298.821457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11298.828484] Lustre: Skipped 1 previous similar message [11304.896762] Lustre: server umount lustre-MDT0000 complete [11309.020211] LustreError: 312254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787472325 with bad export cookie 8943804976799047833 [11309.031455] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11309.411495] Lustre: server umount lustre-MDT0001 complete [11312.906559] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11314.925780] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:05:29 (1787472329) [11323.565182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11324.018292] LustreError: 316544:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11324.044510] LustreError: 316544:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11324.146566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11329.508317] LustreError: 316545:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11330.036321] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11333.604863] LustreError: 316544:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11338.732344] LustreError: 316545:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11340.762332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11341.216340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11346.750369] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11356.352201] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11363.576576] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11370.939094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11371.436532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11378.204883] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11386.964240] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11395.404725] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11395.771170] Lustre: lustre-OST0001: new disk, initializing [11395.777520] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11395.800306] Lustre: 319380:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11396.007180] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11403.421609] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11403.934633] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11403.947915] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11404.010026] LustreError: 319758:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11405.026811] LustreError: 319758:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11405.041616] LustreError: 319758:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11406.060169] LustreError: 319763:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11408.162267] LustreError: 319758:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11408.187641] LustreError: 319758:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11412.344854] LustreError: 319763:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11412.365477] LustreError: 319763:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11415.877157] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11416.173177] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11420.579746] LustreError: 319763:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11420.591394] LustreError: 319763:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11422.787222] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff899745454800.ost_server_uuid 50 [11424.340314] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff899745454800.ost_server_uuid in FULL state after 0 sec [11431.460917] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff899745454800.ost_server_uuid 50 [11432.998641] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff899745454800.ost_server_uuid in IDLE state after 0 sec [11435.039041] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11450.343185] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11450.344177] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11450.357015] Lustre: Skipped 1 previous similar message [11450.362763] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11450.373274] Lustre: Skipped 2 previous similar messages [11454.099863] Lustre: server umount lustre-OST0001 complete [11455.468050] LustreError: 320916:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11455.483148] LustreError: 320916:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11460.586259] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11460.587589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11460.601786] Lustre: Skipped 1 previous similar message [11460.613591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11460.625968] Lustre: Skipped 2 previous similar messages [11465.743697] Lustre: server umount lustre-OST0000 complete [11470.404212] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11470.446320] Lustre: Skipped 2 previous similar messages [11477.003764] Lustre: server umount lustre-MDT0000 complete [11480.546605] LustreError: 316544:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11480.571639] LustreError: 316544:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11481.707481] LustreError: 316525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787472498 with bad export cookie 8943804976799048715 [11481.717233] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11481.737507] LustreError: 316525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11482.268622] Lustre: server umount lustre-MDT0001 complete [11496.294387] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [11499.787385] Key type lgssc unregistered [11500.175925] LNet: 322076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11500.193924] LNetError: 322076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11501.226154] LNet: Removed LNI 192.168.201.152@tcp [11502.327345] Key type .llcrypt unregistered [11502.332957] Key type ._llcrypt unregistered [11532.152552] Key type ._llcrypt registered [11532.157218] Key type .llcrypt registered [11532.286168] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [11533.624245] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11533.729690] alg: No test for adler32 (adler32-zlib) [11534.797981] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [11535.020751] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [11536.680496] Key type lgssc registered [11537.954399] Lustre: Echo OBD driver; http://www.lustre.org/ [11546.619804] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11546.634466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11548.113806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11552.830504] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11557.349667] LustreError: 323498:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11557.371582] LustreError: 323498:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11562.352743] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11562.470001] LustreError: 323499:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11567.589452] LustreError: 323498:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11571.827216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11572.407608] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11576.602496] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11588.637896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11589.204365] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11593.279348] LustreError: 325179:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11593.313335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11597.362338] LustreError: 325181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11597.375225] LustreError: 325181:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11599.461788] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11607.537756] LustreError: 325182:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11607.567845] LustreError: 325182:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11609.034108] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11617.433233] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11617.847983] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11622.904769] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11625.526632] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11635.331900] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11645.008589] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11645.473580] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11756.004492] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11756.023208] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11756.053620] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11760.948666] Lustre: server umount lustre-OST0001 complete [11761.135906] LustreError: 325179:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11761.177255] LustreError: 325179:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11771.362079] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11771.366110] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11771.375735] Lustre: Skipped 1 previous similar message [11771.377819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11771.392347] LustreError: Skipped 1 previous similar message [11771.394356] Lustre: Skipped 1 previous similar message [11774.718391] Lustre: server umount lustre-OST0000 complete [11781.604460] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11781.611573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11781.628537] Lustre: Skipped 2 previous similar messages [11781.652670] Lustre: Skipped 1 previous similar message [11786.721440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11791.842061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11791.847793] Lustre: Skipped 2 previous similar messages [11793.382897] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11793.584934] Lustre: server umount lustre-MDT0000 complete [11796.961887] LustreError: 323493:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11796.975418] LustreError: 323493:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11798.391264] LustreError: 327185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787472815 with bad export cookie 1245017950999078726 [11798.400235] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11798.402237] LustreError: 327185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11798.769323] Lustre: server umount lustre-MDT0001 complete [11814.683415] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [11819.083134] Key type lgssc unregistered [11819.336357] LNet: 328454:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11819.363332] LNetError: 328454:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11820.404254] LNet: Removed LNI 192.168.201.152@tcp [11821.376491] Key type .llcrypt unregistered [11821.378706] Key type ._llcrypt unregistered [11876.335122] Key type ._llcrypt registered [11876.336145] Key type .llcrypt registered [11876.530773] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [11878.052752] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11878.293880] alg: No test for adler32 (adler32-zlib) [11879.838427] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [11880.323984] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [11882.160410] Key type lgssc registered [11883.705882] Lustre: Echo OBD driver; http://www.lustre.org/ [11898.546509] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11898.581885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11900.456949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11907.144490] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11910.643902] LustreError: 330750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11910.669743] LustreError: 330750:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11915.759567] LustreError: 330750:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11917.439344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11917.959190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11923.811798] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11928.013710] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11936.554975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11937.088615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11942.213330] LustreError: 332240:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11944.064811] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11945.270362] LustreError: 332515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11945.296891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11945.311400] LustreError: 332515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11950.574507] LustreError: 332241:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11950.599805] LustreError: 332241:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11954.506344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11954.669894] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11959.833586] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11961.840561] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [11974.215207] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11981.355566] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11999.714907] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11999.724541] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11999.747180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12000.740320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12000.741784] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12000.749218] Lustre: Skipped 1 previous similar message [12004.936627] Lustre: server umount lustre-MDT0000 complete [12005.889685] LustreError: 330744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12005.941378] LustreError: 330744:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12010.593698] LustreError: 330731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787473027 with bad export cookie 18148391053000890165 [12010.605408] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12010.619434] LustreError: 330731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12010.980545] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12010.987348] Lustre: Skipped 2 previous similar messages [12011.107760] Lustre: server umount lustre-MDT0001 complete [12017.909940] Lustre: server umount lustre-OST0000 complete [12023.751181] Lustre: server umount lustre-OST0001 complete [12045.484509] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [12058.657128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12059.103178] LustreError: 336018:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12059.131176] LustreError: 336018:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12059.215892] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12065.093204] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12075.647079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12076.310250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12081.605425] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12084.923485] Lustre: 337160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12092.996305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12093.599697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12100.736212] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12101.811880] LustreError: 337515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12101.843794] LustreError: 337515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12104.915957] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12111.109644] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12116.486588] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12119.450275] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12128.069325] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12132.362049] Lustre: 339033:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12142.055264] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12142.067352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12142.080788] Lustre: Skipped 2 previous similar messages [12142.085718] Lustre: Skipped 4 previous similar messages [12144.672195] Lustre: server umount lustre-MDT0000 complete [12149.119510] LustreError: 336000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787473165 with bad export cookie 18148391053000891866 [12149.126295] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12149.138108] LustreError: 336000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12149.599048] Lustre: server umount lustre-MDT0001 complete [12163.374837] Lustre: server umount lustre-OST0000 complete [12179.290343] Lustre: server umount lustre-OST0001 complete [12184.480823] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12186.653450] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12188.558640] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:20:02 (1787473202) [12220.507307] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [12231.300228] loop0: detected capacity change from 0 to 1099511627776 [12232.093642] SGI XFS with ACLs, security attributes, quota, no debug enabled [12232.099856] *** VALIDATE xfs *** [12232.108666] XFS (loop0): Mounting V5 Filesystem [12232.158042] XFS (loop0): Ending clean mount [12233.688706] loop1: detected capacity change from 0 to 18691697672192 [12381.115125] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [12395.344518] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [12440.673864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12440.997513] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12441.030636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12441.119539] Lustre: lustre-MDT0000: new disk, initializing [12441.260234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12441.267691] Lustre: Skipped 1 previous similar message [12441.290181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12447.912631] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12461.053275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12461.230461] Lustre: 344214:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12461.344254] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12461.349809] Lustre: Skipped 1 previous similar message [12461.447525] Lustre: lustre-MDT0001: new disk, initializing [12461.678188] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12461.693600] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12467.383541] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12479.599721] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12486.985497] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12495.532506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12496.048899] Lustre: lustre-OST0000: new disk, initializing [12496.058046] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12496.079184] Lustre: 346181:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12497.403948] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12497.416280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12497.598972] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12505.091411] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12517.696554] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12523.546806] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12523.842429] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12529.749750] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12530.084086] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12533.218126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12533.224478] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12533.232730] Lustre: Skipped 1 previous similar message [12533.239372] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12533.246333] Lustre: Skipped 2 previous similar messages [12538.829417] Lustre: server umount lustre-OST0000 complete [12543.864077] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12543.890413] Lustre: Skipped 1 previous similar message [12543.899225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12543.910067] Lustre: Skipped 1 previous similar message [12548.580636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12548.587019] LustreError: Skipped 1 previous similar message [12553.252859] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12553.267721] Lustre: Skipped 4 previous similar messages [12558.313527] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12558.563694] Lustre: server umount lustre-MDT0000 complete [12563.426592] LustreError: 344219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12563.442654] LustreError: 344219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [12563.781784] LustreError: 344205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787473580 with bad export cookie 18148391053000893420 [12563.785396] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12563.795292] LustreError: 344205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12564.233763] Lustre: server umount lustre-MDT0001 complete [12574.895945] XFS (loop0): Unmounting Filesystem [12582.199458] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:26:35 (1787473595) [12594.501037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12595.267479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12595.278534] Lustre: Skipped 2 previous similar messages [12602.061298] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12614.638134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12622.574412] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12632.008744] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12640.042738] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12648.243414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12655.860180] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12666.164148] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12681.714395] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12681.715970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12681.730047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12681.730060] Lustre: Skipped 2 previous similar messages [12681.771395] Lustre: Skipped 2 previous similar messages [12684.822373] Lustre: server umount lustre-OST0000 complete [12704.224564] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12704.373856] Lustre: server umount lustre-MDT0000 complete [12706.284496] LustreError: 348744:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12706.323062] LustreError: 348744:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12708.819302] LustreError: 348726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787473725 with bad export cookie 18148391053000894302 [12708.834950] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12709.264920] Lustre: server umount lustre-MDT0001 complete [12725.404328] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [12728.918309] Key type lgssc unregistered [12729.247408] LNet: 352381:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12729.260663] LNetError: 352381:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12729.271230] LNet: Removed LNI 192.168.201.152@tcp [12730.710331] Key type .llcrypt unregistered [12730.711921] Key type ._llcrypt unregistered [12766.121538] Key type ._llcrypt registered [12766.123627] Key type .llcrypt registered [12766.245830] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [12767.957342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12768.130897] alg: No test for adler32 (adler32-zlib) [12769.385256] Lustre: Lustre: Build Version: 2.17.57_65_gc740cad [12769.816267] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [12771.536239] Key type lgssc registered [12773.391821] Lustre: Echo OBD driver; http://www.lustre.org/ [12783.683617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12783.697558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12785.470300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12791.134802] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12795.893881] LustreError: 353953:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12795.945409] LustreError: 353953:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12800.998244] LustreError: 353954:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12803.439393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12803.818110] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12808.908924] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12820.323965] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12827.310296] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12836.213261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12836.612720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12844.633773] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12854.777470] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12865.508418] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12865.522596] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12865.566325] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12866.532295] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12866.547664] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12867.724448] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (stopping) [12867.734235] Lustre: Skipped 1 previous similar message [12870.577386] Lustre: server umount lustre-MDT0001 complete [12871.662764] LustreError: 354681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12871.703745] LustreError: 354681:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12876.776687] LustreError: 353948:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12876.815286] LustreError: 353948:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12881.906388] LustreError: 354681:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12881.926615] LustreError: 354681:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12884.085670] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12884.287698] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12884.297688] Lustre: Found index 1 for lustre-MDT0001, updating log [12884.377943] Lustre: 353943:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12884.897995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12890.123277] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12891.410683] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12893.706561] LustreError: 357488:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12893.713779] LustreError: 357488:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12893.724202] LustreError: 353946:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12893.855372] LustreError: 357497:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -17 [12893.867454] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12893.867454] [12905.446279] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12905.459402] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12905.462570] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12905.512979] Lustre: Skipped 1 previous similar message [12910.565348] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12910.575669] Lustre: Skipped 3 previous similar messages [12918.240428] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12918.514670] Lustre: server umount lustre-MDT0001 complete [12918.866733] LustreError: 353950:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12918.896049] LustreError: 353950:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12931.672048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12931.788227] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12931.794037] Lustre: Found index 1 for lustre-MDT0001, updating log [12931.844270] Lustre: 353942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12932.253108] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12937.713414] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12938.518790] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [12940.291191] LustreError: 358535:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12940.319579] LustreError: 358535:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12940.344319] LustreError: 353946:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12940.353605] LustreError: 353946:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12940.479957] LustreError: 358545:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12940.493735] LustreError: 358545:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12940.512567] LustreError: 358545:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -17 [12940.530448] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.152@tcp [12940.530448] [13250.033723] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13250.034310] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13250.044816] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13250.087472] Lustre: Skipped 4 previous similar messages [13261.281367] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13261.668566] Lustre: server umount lustre-MDT0001 complete [13261.907448] LustreError: 361167:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13261.953511] LustreError: 361167:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [13277.195541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13277.446326] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13277.454202] Lustre: Found index 1 for lustre-MDT0001, updating log [13277.531795] Lustre: 353941:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13277.806647] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13283.244659] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [13283.318626] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13285.361056] LustreError: 366227:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13285.367818] LustreError: 366227:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13285.377481] LustreError: 353946:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13285.381848] LustreError: 353946:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13285.510303] LustreError: 366233:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13285.546902] LustreError: 366233:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -114 [13285.561069] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13285.561069] [13605.345785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13605.354981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13605.365506] Lustre: Skipped 1 previous similar message [13605.369834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13605.376317] Lustre: Skipped 6 previous similar messages [13611.191416] Lustre: server umount lustre-MDT0000 complete [13615.786860] LustreError: 357142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787474632 with bad export cookie 8793682395183211517 [13615.801276] LustreError: 357142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13615.801974] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13616.102657] LustreError: 353948:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13616.103802] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13616.125436] LustreError: 353948:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [13616.148535] Lustre: Skipped 2 previous similar messages [13622.520961] Lustre: server umount lustre-MDT0001 complete [13627.674098] Lustre: server umount lustre-OST0000 complete [13638.750725] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [13650.241720] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [13693.714333] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:45:08 (1787474708) [13700.910494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13701.106772] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13701.140636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13701.220192] Lustre: lustre-MDT0000: new disk, initializing [13701.313544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13701.330427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13705.792883] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [13718.306525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13718.520838] Lustre: 376340:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13718.586505] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13718.590767] Lustre: Skipped 1 previous similar message [13718.649807] Lustre: lustre-MDT0001: new disk, initializing [13718.710482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13718.734462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13718.739407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13724.259877] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [13735.965478] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13742.962212] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13749.983280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13750.352895] Lustre: lustre-OST0000: new disk, initializing [13750.356739] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13750.366291] Lustre: 378310:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13750.478295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13756.961984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13756.966742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13757.141330] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13757.501348] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [13768.793549] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13777.892774] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13777.899425] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13777.902611] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13777.906789] Lustre: Skipped 2 previous similar messages [13777.925876] Lustre: Skipped 9 previous similar messages [13783.292786] Lustre: server umount lustre-MDT0000 complete [13787.935986] LustreError: 376334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787474804 with bad export cookie 8793682395183213470 [13787.942395] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13787.958829] LustreError: 376334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13788.183208] LustreError: 376346:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13788.202783] LustreError: 376346:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13788.416499] Lustre: server umount lustre-MDT0001 complete [13795.441262] Lustre: server umount lustre-OST0000 complete [13834.404235] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [13846.199105] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [13886.712808] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13637 sec ========= 04:48:20 (1787474900) [13888.934716] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:48:22 (1787474902) === [13894.125914] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:48:27 (1787474907) === [13939.806675] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [13942.779593] Key type lgssc unregistered [13943.090757] LNet: 384681:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13943.095778] LNetError: 384681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13944.173714] LNet: Removed LNI 192.168.201.152@tcp [13945.472647] Key type .llcrypt unregistered [13945.476589] Key type ._llcrypt unregistered