[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 456291610 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002305] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008014] pid_max: default: 32768 minimum: 301 [ 0.009117] LSM: Security Framework initializing [ 0.010041] Yama: becoming mindful. [ 0.011023] SELinux: Initializing. [ 0.012070] *** VALIDATE selinux *** [ 0.020528] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025294] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026131] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027097] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028123] *** VALIDATE tmpfs *** [ 0.030263] *** VALIDATE proc *** [ 0.031174] *** VALIDATE cgroup *** [ 0.032000] *** VALIDATE cgroup2 *** [ 0.033066] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034147] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036023] Spectre V2 : User space: Vulnerable [ 0.037007] Speculative Store Bypass: Vulnerable [ 0.040058] debug: unmapping init [mem 0xffffffff9e259000-0xffffffff9e260fff] [ 0.042242] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043728] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044019] ... version: 2 [ 0.045010] ... bit width: 48 [ 0.046012] ... generic registers: 4 [ 0.047008] ... value mask: 0000ffffffffffff [ 0.048007] ... max period: 00007fffffffffff [ 0.048970] ... fixed-purpose events: 3 [ 0.049005] ... event mask: 000000070000000f [ 0.051206] rcu: Hierarchical SRCU implementation. [ 0.053135] smp: Bringing up secondary CPUs ... [ 0.054400] x86: Booting SMP configuration: [ 0.055018] .... node #0, CPUs: #1 #2 #3 [ 0.058190] smp: Brought up 1 node, 4 CPUs [ 0.060019] smpboot: Max logical packages: 1 [ 0.061032] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.223020] node 0 deferred pages initialised in 160ms [ 0.227174] devtmpfs: initialized [ 0.228235] x86/mm: Memory block size: 128MB [ 0.230686] gcov: version magic: 0x41383552 [ 0.231711] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.234122] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.236334] pinctrl core: initialized pinctrl subsystem [ 0.238168] [ 0.238675] ************************************************************* [ 0.241011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.243011] ** ** [ 0.245010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.246008] ** ** [ 0.248015] ** This means that this kernel is built to expose internal ** [ 0.250008] ** IOMMU data structures, which may compromise security on ** [ 0.252014] ** your system. ** [ 0.253006] ** ** [ 0.255008] ** If you see this message and you are not debugging the ** [ 0.257006] ** kernel, report this immediately to your vendor! ** [ 0.259012] ** ** [ 0.261008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.263012] ************************************************************* [ 0.265901] NET: Registered protocol family 16 [ 0.267489] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.270074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.273060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.277018] cpuidle: using governor menu [ 0.278788] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.281802] PCI: Using configuration type 1 for base access [ 0.283168] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.291285] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.294030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.297167] cryptd: max_cpu_qlen set to 1000 [ 0.301297] ACPI: Added _OSI(Module Device) [ 0.303014] ACPI: Added _OSI(Processor Device) [ 0.304008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.305010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.309403] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.316360] ACPI: Interpreter enabled [ 0.318065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.319009] ACPI: Using IOAPIC for interrupt routing [ 0.323165] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.326424] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.335660] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.338038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.340020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.343094] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.347505] acpiphp: Slot [2] registered [ 0.348129] acpiphp: Slot [5] registered [ 0.350142] acpiphp: Slot [6] registered [ 0.351102] acpiphp: Slot [7] registered [ 0.353123] acpiphp: Slot [8] registered [ 0.354145] acpiphp: Slot [9] registered [ 0.356098] acpiphp: Slot [10] registered [ 0.357060] acpiphp: Slot [3] registered [ 0.358095] acpiphp: Slot [4] registered [ 0.360080] acpiphp: Slot [11] registered [ 0.361161] acpiphp: Slot [12] registered [ 0.363083] acpiphp: Slot [13] registered [ 0.364083] acpiphp: Slot [14] registered [ 0.367130] acpiphp: Slot [15] registered [ 0.369225] acpiphp: Slot [16] registered [ 0.371111] acpiphp: Slot [17] registered [ 0.372044] acpiphp: Slot [18] registered [ 0.373019] acpiphp: Slot [19] registered [ 0.374064] acpiphp: Slot [20] registered [ 0.374964] acpiphp: Slot [21] registered [ 0.376099] acpiphp: Slot [22] registered [ 0.377048] acpiphp: Slot [23] registered [ 0.379146] acpiphp: Slot [24] registered [ 0.381266] acpiphp: Slot [25] registered [ 0.384128] acpiphp: Slot [26] registered [ 0.386106] acpiphp: Slot [27] registered [ 0.388084] acpiphp: Slot [28] registered [ 0.390089] acpiphp: Slot [29] registered [ 0.391111] acpiphp: Slot [30] registered [ 0.393085] acpiphp: Slot [31] registered [ 0.394060] PCI host bridge to bus 0000:00 [ 0.396016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.398030] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.401017] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.404019] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.407020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.410025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.412193] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.416559] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.420083] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.430886] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.437036] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.440030] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.443033] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.445025] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.448146] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.450799] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.452042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.455940] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.460017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.474017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.479018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.487558] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.496026] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.507024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.525027] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.548673] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.557018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.562018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.578015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.587636] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.594000] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.599013] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.617023] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.631734] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.638014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.644010] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.658036] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.666547] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.671011] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.675012] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.690014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.698973] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.704015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.710019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.725016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.735239] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.737395] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.739324] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.741322] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.743290] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.749140] iommu: Default domain type: Passthrough [ 0.750346] SCSI subsystem initialized [ 0.752146] ACPI: bus type USB registered [ 0.752989] usbcore: registered new interface driver usbfs [ 0.755063] usbcore: registered new interface driver hub [ 0.756051] usbcore: registered new device driver usb [ 0.757155] pps_core: LinuxPPS API ver. 1 registered [ 0.759010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.762129] PTP clock support registered [ 0.764107] EDAC MC: Ver: 3.0.0 [ 0.766239] PCI: Using ACPI for IRQ routing [ 0.767525] NetLabel: Initializing [ 0.768011] NetLabel: domain hash size = 128 [ 0.770009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.771075] NetLabel: unlabeled traffic allowed by default [ 0.773280] vgaarb: loaded [ 0.774319] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.776013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.781612] clocksource: Switched to clocksource kvm-clock [ 0.894196] VFS: Disk quotas dquot_6.6.0 [ 0.895945] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.898921] *** VALIDATE ramfs *** [ 0.900396] *** VALIDATE hugetlbfs *** [ 0.902443] pnp: PnP ACPI init [ 0.905320] pnp: PnP ACPI: found 6 devices [ 0.927799] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.931480] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.933670] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.936538] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.940095] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.943908] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.948901] NET: Registered protocol family 2 [ 0.952151] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.958340] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.963822] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.969347] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.972517] TCP: Hash tables configured (established 65536 bind 65536) [ 0.976368] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.981497] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.984078] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.986531] NET: Registered protocol family 1 [ 0.988489] RPC: Registered named UNIX socket transport module. [ 0.990007] RPC: Registered udp transport module. [ 0.991328] RPC: Registered tcp transport module. [ 0.992717] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.994857] NET: Registered protocol family 44 [ 0.996449] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.998431] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.000420] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.002517] PCI: CLS 0 bytes, default 64 [ 1.003940] Unpacking initramfs... [ 2.653897] debug: unmapping init [mem 0xffff997a3cc54000-0xffff997a3ffbffff] [ 2.660227] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.662813] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.666901] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.269444] Initialise system trusted keyrings [ 3.271435] Key type blacklist registered [ 3.273528] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.282620] zbud: loaded [ 3.285207] *** VALIDATE nfs *** [ 3.286502] *** VALIDATE nfs4 *** [ 3.288205] pstore: using deflate compression [ 3.292253] Platform Keyring initialized [ 3.408890] NET: Registered protocol family 38 [ 3.412091] Key type asymmetric registered [ 3.413337] Asymmetric key parser 'x509' registered [ 3.415875] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.419150] io scheduler mq-deadline registered [ 3.420472] io scheduler kyber registered [ 3.422399] io scheduler bfq registered [ 3.425040] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.428564] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.432586] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.436248] ACPI: Power Button [PWRF] [ 3.443700] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.453170] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.468618] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.478826] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.508505] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.536760] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.566539] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.573954] Non-volatile memory driver v1.3 [ 3.576838] Linux agpgart interface v0.103 [ 3.607887] virtio_blk virtio1: [vda] 146144 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.610820] vda: detected capacity change from 0 to 74825728 [ 3.623368] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.626423] vdb: detected capacity change from 0 to 1073741824 [ 3.641837] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.645788] vdc: detected capacity change from 0 to 2621440000 [ 3.668869] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.672627] vdd: detected capacity change from 0 to 2621440000 [ 3.689338] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.691807] vde: detected capacity change from 0 to 4294967296 [ 3.709356] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.713399] vdf: detected capacity change from 0 to 4294967296 [ 3.726924] libphy: Fixed MDIO Bus: probed [ 3.734433] usbcore: registered new interface driver usbserial_generic [ 3.737618] usbserial: USB Serial support registered for generic [ 3.743188] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.749236] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.751777] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.754250] mousedev: PS/2 mouse device common for all mice [ 3.761351] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.765898] rtc_cmos 00:05: RTC can wake from S4 [ 3.769960] rtc_cmos 00:05: registered as rtc0 [ 3.773107] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.776489] intel_pstate: CPU model not supported [ 3.780082] hid: raw HID events driver (C) Jiri Kosina [ 3.781122] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.784267] usbcore: registered new interface driver usbhid [ 3.784275] usbhid: USB HID core driver [ 3.784423] drop_monitor: Initializing network drop monitor service [ 3.786764] Initializing XFRM netlink socket [ 3.800621] NET: Registered protocol family 10 [ 3.803440] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.804106] Segment Routing with IPv6 [ 3.811953] NET: Registered protocol family 17 [ 3.814150] mpls_gso: MPLS GSO support [ 3.820473] RAS: Correctable Errors collector initialized. [ 3.822223] AVX version of gcm_enc/dec engaged. [ 3.824968] AES CTR mode by8 optimization enabled [ 3.945922] sched_clock: Marking stable (3945838585, 0)->(4731519704, -785681119) [ 3.950422] registered taskstats version 1 [ 3.952944] Loading compiled-in X.509 certificates [ 3.955617] zswap: loaded using pool lzo/zbud [ 3.986289] Key type big_key registered [ 4.001265] Key type encrypted registered [ 4.002883] ima: No TPM chip found, activating TPM-bypass! [ 4.005770] ima: Allocated hash algorithm: sha1 [ 4.007836] ima: No architecture policies found [ 4.010541] evm: Initialising EVM extended attributes: [ 4.012853] evm: security.selinux [ 4.014634] evm: security.ima [ 4.016180] evm: security.capability [ 4.017768] evm: HMAC attrs: 0x1 [ 4.023422] rtc_cmos 00:05: setting system clock to 2026-09-01 14:03:55 UTC (1788271435) [ 4.033847] debug: unmapping init [mem 0xffffffff9f203000-0xffffffff9f3fffff] [ 4.038580] debug: unmapping init [mem 0xffffffff9df82000-0xffffffff9e258fff] [ 4.048530] Write protecting the kernel read-only data: 28672k [ 4.055202] debug: unmapping init [mem 0xffffffff9c603000-0xffffffff9c7fffff] [ 4.064513] debug: unmapping init [mem 0xffffffff9cf14000-0xffffffff9cffffff] [ 4.145731] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.159822] systemd[1]: Detected virtualization kvm. [ 4.163235] systemd[1]: Detected architecture x86-64. [ 4.165682] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.209874] systemd[1]: No hostname configured. [ 4.213198] systemd[1]: Set hostname to . [ 4.215411] random: systemd: uninitialized urandom read (16 bytes read) [ 4.218896] systemd[1]: Initializing machine ID from random generator. [ 4.289510] random: ln: uninitialized urandom read (6 bytes read) [ 4.496248] random: systemd: uninitialized urandom read (16 bytes read) [ 4.499565] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.505286] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.514056] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.725127] device-mapper: uevent: version 1.0.3 [ 5.730179] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 7.156081] random: fast init done [ 7.178306] virtio_net virtio0 ens2: renamed from eth0 [ 7.315354] scsi host0: ata_piix [ 7.419172] scsi host1: ata_piix [ 7.426638] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.435817] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 12.528975] random: crng init done [ 12.532220] random: 7 urandom warning(s) missed due to ratelimiting [ 13.411853] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 15.242615] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 17.616163] printk: systemd: 26 output lines suppressed due to ratelimiting [ 18.367827] SELinux: Disabled at runtime. [ 18.474788] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.490562] systemd[1]: Detected virtualization kvm. [ 18.495140] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 19.658483] systemd[1]: initrd-switch-root.service: Succeeded. [ 19.674574] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 19.695483] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 19.723886] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 19.737731] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 19.757680] systemd[1]: Starting Journal Service... Starting Journal Service... [ 19.777257] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Local Encrypted Volumes. Starting udev Coldplug all Devices... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped targ[ 20.166789] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS et Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 20.715547] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.352538] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.371542] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.689344] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.722364] EDAC sbridge: Ver: 1.1.2 [ 24.395940] Key type dns_resolver registered [ 24.973533] NFS: Registering the id_resolver key type [ 24.975623] Key type id_resolver registered [ 24.977253] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg453-server login: [ 87.883419] libcfs: loading out-of-tree module taints kernel. [ 87.940923] Key type ._llcrypt registered [ 87.945363] Key type .llcrypt registered [ 88.044513] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 110.452042] hrtimer: interrupt took 3228854 ns [ 112.883284] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 115.616773] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 115.667946] alg: No test for adler32 (adler32-zlib) [ 117.709710] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 119.292674] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 121.255182] Key type lgssc registered [ 123.605313] Lustre: Echo OBD driver; http://www.lustre.org/ [ 149.588281] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 185.058341] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 194.604614] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 194.641150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 195.869741] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 195.905803] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 195.991330] Lustre: lustre-MDT0000: new disk, initializing [ 196.047990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 196.063161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 199.392240] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.569356] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 209.636200] Lustre: 6518:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 209.660444] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 209.664571] Lustre: Skipped 1 previous similar message [ 209.731622] Lustre: lustre-MDT0001: new disk, initializing [ 209.783592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 209.803738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 209.816857] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 212.956389] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 216.737946] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 224.473106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 224.698880] Lustre: lustre-OST0000: new disk, initializing [ 224.702324] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 224.709503] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 224.767949] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 227.655585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 227.666280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 227.713595] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 229.547523] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 241.845383] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 241.985370] Lustre: lustre-OST0001: new disk, initializing [ 241.990597] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 241.999066] Lustre: 9525:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 242.052219] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 247.386303] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 250.380806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 250.398286] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 250.432027] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 257.189440] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 262.846598] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 267.937650] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing check_logdir /tmp/testlogs/ [ 271.992457] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing yml_node [ 275.722929] Lustre: DEBUG MARKER: Client: 2.17.57.64 [ 277.899189] Lustre: DEBUG MARKER: MDS: 2.17.57.64 [ 280.024584] Lustre: DEBUG MARKER: OSS: 2.17.57.64 [ 281.221989] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 1 10:08:31 EDT 2026 [ 294.515857] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 295.601373] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 311.779836] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 311.782686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 311.795118] Lustre: Skipped 3 previous similar messages [ 311.807052] Lustre: Skipped 3 previous similar messages [ 315.782429] Lustre: server umount lustre-MDT0000 complete [ 319.000188] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271750 with bad export cookie 4716768639026637497 [ 319.004931] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 319.011721] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 322.015944] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 322.016927] LustreError: 6523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 322.036731] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 322.062349] LustreError: 6523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 322.085448] Lustre: Skipped 1 previous similar message [ 324.069587] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 324.071122] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 324.080247] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 324.089632] Lustre: Skipped 1 previous similar message [ 325.482624] Lustre: server umount lustre-MDT0001 complete [ 328.667161] Lustre: server umount lustre-OST0000 complete [ 332.119359] Lustre: server umount lustre-OST0001 complete [ 338.404538] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 347.435717] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 377.934433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 378.110775] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 378.140888] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 378.213192] Lustre: lustre-MDT0000: new disk, initializing [ 378.313081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 378.331443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 382.381511] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 391.500216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 391.588679] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 391.641727] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 391.644265] Lustre: Skipped 1 previous similar message [ 391.698233] Lustre: lustre-MDT0001: new disk, initializing [ 391.782269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 391.831720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 391.846807] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 395.299547] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 403.270399] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 408.079864] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 413.204869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 413.408262] Lustre: lustre-OST0000: new disk, initializing [ 413.415400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 413.423355] Lustre: 17043:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 413.511646] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 415.345900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 415.353158] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 415.437148] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 418.062510] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 425.701170] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 429.374753] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 429.538450] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 432.858838] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 433.005614] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 436.195446] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 436.197043] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 436.204432] LustreError: Skipped 1 previous similar message [ 436.205164] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 436.230106] Lustre: Skipped 2 previous similar messages [ 440.718561] Lustre: server umount lustre-OST0000 complete [ 446.433945] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 446.435236] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 446.440438] Lustre: Skipped 1 previous similar message [ 446.449665] Lustre: Skipped 2 previous similar messages [ 449.940399] Lustre: server umount lustre-MDT0000 complete [ 451.557470] LustreError: 15081:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 451.577234] LustreError: 15081:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 452.670417] LustreError: 15065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271884 with bad export cookie 4716768639026639030 [ 452.682415] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 452.915924] Lustre: server umount lustre-MDT0001 complete [ 456.670241] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:11:27 (1788271887) [ 462.151321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 462.543106] LustreError: 19307:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 462.621184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 466.550784] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 467.938830] LustreError: 19308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 474.639288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.329717] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 486.283673] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 491.857911] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 497.916668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 503.443973] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 509.176574] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 512.450872] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 518.628068] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 518.629152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 518.644411] Lustre: Skipped 1 previous similar message [ 518.660709] Lustre: Skipped 2 previous similar messages [ 524.203521] Lustre: server umount lustre-MDT0000 complete [ 527.168201] LustreError: 19287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788271958 with bad export cookie 4716768639026639912 [ 527.168623] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 527.180761] LustreError: 19287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 527.434741] Lustre: server umount lustre-MDT0001 complete [ 541.078547] Lustre: server umount lustre-OST0000 complete [ 557.577935] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 564.781552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 565.099908] LustreError: 23684:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 565.117658] LustreError: 23684:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 565.155937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 565.162392] Lustre: Skipped 2 previous similar messages [ 568.873866] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 575.515339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 578.712947] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 580.956291] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 586.351882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 592.265195] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 599.506544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 599.620103] Lustre: lustre-OST0001: new disk, initializing [ 599.624667] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 599.629656] Lustre: 25998:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 601.137991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 601.145481] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 601.212756] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 604.553423] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 612.743531] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 615.665424] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 626.664338] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 626.667463] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 626.676224] Lustre: Skipped 4 previous similar messages [ 626.679299] Lustre: Skipped 5 previous similar messages [ 630.524569] Lustre: server umount lustre-MDT0000 complete [ 631.781909] LustreError: 23680:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 631.800036] LustreError: 23680:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 633.181972] LustreError: 23665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272064 with bad export cookie 4716768639026640759 [ 633.187211] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 633.189183] LustreError: 23665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 633.426258] Lustre: server umount lustre-MDT0001 complete [ 646.188317] Lustre: server umount lustre-OST0000 complete [ 657.860951] Lustre: server umount lustre-OST0001 complete [ 667.958688] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 674.008694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 674.295954] LustreError: 29016:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 676.848394] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 681.609467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 684.031838] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 685.708628] Lustre: 30163:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 689.856975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 693.771275] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 698.542810] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.698648] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 698.704626] Lustre: Skipped 6 previous similar messages [ 702.177242] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 707.373448] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 709.635456] Lustre: 32043:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 715.238402] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 715.239836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 715.250840] Lustre: Skipped 2 previous similar messages [ 715.261303] Lustre: Skipped 4 previous similar messages [ 719.573185] Lustre: server umount lustre-MDT0000 complete [ 722.602321] LustreError: 31355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272154 with bad export cookie 4716768639026642047 [ 722.606250] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 722.617311] LustreError: 31355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 722.976840] Lustre: server umount lustre-MDT0001 complete [ 736.281732] Lustre: server umount lustre-OST0000 complete [ 749.205452] Lustre: server umount lustre-OST0001 complete [ 755.683649] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:16:26 (1788272186) [ 773.885922] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 780.778211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 781.104941] LustreError: 34841:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 781.118807] LustreError: 34841:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 784.163856] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 790.288169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 794.251293] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 796.408426] Lustre: 35987:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 800.983659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 805.833879] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 811.577025] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 816.125173] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 822.001906] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 824.350316] Lustre: 37867:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 830.441613] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 830.451022] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 830.454922] Lustre: Skipped 3 previous similar messages [ 830.470284] Lustre: Skipped 3 previous similar messages [ 833.677557] Lustre: server umount lustre-MDT0000 complete [ 836.269448] LustreError: 34824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272267 with bad export cookie 4716768639026643153 [ 836.276112] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 836.280632] LustreError: 34824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 836.474759] Lustre: server umount lustre-MDT0001 complete [ 848.922648] Lustre: server umount lustre-OST0000 complete [ 860.908484] Lustre: server umount lustre-OST0001 complete [ 875.762703] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:18:23 (1788272303) [ 905.873594] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 916.018024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 916.481681] LustreError: 40031:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 916.502568] LustreError: 40031:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 921.062770] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 930.045572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 934.728678] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 937.989809] Lustre: 41179:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 943.786085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 949.620286] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 958.386276] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 958.641169] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 958.645087] Lustre: Skipped 7 previous similar messages [ 964.051086] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 971.356874] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 974.558024] Lustre: 43061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 975.592809] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 975.600365] Lustre: Skipped 1 previous similar message [ 981.441120] Lustre: server umount lustre-MDT0000 complete [ 981.472863] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 981.492187] Lustre: Skipped 4 previous similar messages [ 985.765345] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272417 with bad export cookie 4716768639026644266 [ 985.768957] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 985.785288] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 986.628355] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 986.636288] Lustre: Skipped 1 previous similar message [ 992.157522] Lustre: server umount lustre-MDT0001 complete [ 1000.692834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1011.169379] LustreError: 3648:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff997aa910f480 x1875138629167744/t0(0) o250->MGC192.168.204.153@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1016.307486] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1020.783728] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1025.042208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1029.009744] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1030.644720] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1030.652683] Lustre: Skipped 1 previous similar message [ 1036.297946] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1037.928703] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1043.260705] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1044.824765] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1051.133917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1056.657914] Lustre: server umount lustre-MDT0000 complete [ 1059.820748] LustreError: 41181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272491 with bad export cookie 4716768639026645393 [ 1059.825282] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1059.835519] LustreError: 41181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1060.222943] Lustre: server umount lustre-MDT0001 complete [ 1074.329451] Lustre: server umount lustre-OST0000 complete [ 1077.215745] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788272492/real 1788272492] req@ffff997981bc9180 x1875138629200640/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788272508 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1078.002850] Lustre: server umount lustre-OST0001 complete [ 1085.626989] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:21:55 (1788272515) [ 1100.140540] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1108.896454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1113.663358] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1121.729978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1125.936911] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1128.861521] Lustre: 48824:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1135.999380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1142.607409] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1150.431587] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1157.195878] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1165.181543] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1169.682927] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1183.712645] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1183.720968] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1183.730665] Lustre: Skipped 7 previous similar messages [ 1189.283858] Lustre: server umount lustre-MDT0000 complete [ 1189.866897] LustreError: 48748:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1189.887410] LustreError: 48748:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 27 previous similar messages [ 1194.076516] LustreError: 47651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788272625 with bad export cookie 4716768639026646191 [ 1194.082290] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1194.103013] LustreError: 47651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1194.993438] Lustre: server umount lustre-MDT0001 complete [ 1210.054446] Lustre: server umount lustre-OST0000 complete [ 1211.360555] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788272626/real 1788272626] req@ffff997ac03bce00 x1875138629256960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788272642 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1214.074344] Lustre: server umount lustre-OST0001 complete [ 1224.483618] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:24:14 (1788272654) [ 1227.304301] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1229.502223] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:24:19 (1788272659) [ 1239.637849] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 1247.250466] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1268.600862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1268.842335] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1268.917754] Lustre: lustre-MDT0000: new disk, initializing [ 1269.026179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1273.268630] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1281.858752] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1288.023240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1288.234922] Lustre: 54902:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1289.403512] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1289.445717] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1294.350520] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1302.946066] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1305.090778] Lustre: DEBUG MARKER: create test files [ 1340.904440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1340.907087] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1340.916313] Lustre: Skipped 16 previous similar messages [ 1342.526648] Lustre: server umount lustre-OST0000 complete [ 1346.688764] Lustre: server umount lustre-MDT0000 complete [ 1358.229802] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 1361.235439] Key type lgssc unregistered [ 1361.604847] LNet: 56537:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1361.618100] LNetError: 56537:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1361.638942] LNet: Removed LNI 192.168.204.153@tcp [ 1362.855253] Key type .llcrypt unregistered [ 1362.857407] Key type ._llcrypt unregistered [ 1389.459782] Key type ._llcrypt registered [ 1389.461167] Key type .llcrypt registered [ 1389.541544] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1390.369495] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1390.383894] alg: No test for adler32 (adler32-zlib) [ 1391.460572] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 1391.732695] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 1393.415184] Key type lgssc registered [ 1394.366705] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1402.382255] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1402.428788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1403.975149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1408.474568] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1415.515227] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1421.394087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1421.770312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1427.222136] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1428.984339] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1429.133782] Lustre: 59249:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1429.139047] Lustre: 59249:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1429.143442] Lustre: 59249:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1429.147199] Lustre: 59249:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1429.152223] Lustre: 59249:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1429.155742] Lustre: 59249:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1434.530812] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1439.139586] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1445.042967] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1453.420118] Lustre: server umount lustre-OST0000 complete [ 1467.539404] Lustre: server umount lustre-MDT0000 complete [ 1477.197237] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 1479.719812] Key type lgssc unregistered [ 1480.038485] LNet: 60863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1480.044964] LNetError: 60863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1480.064034] LNet: Removed LNI 192.168.204.153@tcp [ 1480.864133] Key type .llcrypt unregistered [ 1480.866370] Key type ._llcrypt unregistered [ 1512.774521] Key type ._llcrypt registered [ 1512.778573] Key type .llcrypt registered [ 1512.888786] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1514.341278] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1514.388829] alg: No test for adler32 (adler32-zlib) [ 1515.447490] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 1515.653455] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 1517.287227] Key type lgssc registered [ 1518.567761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1525.856917] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1525.883312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1527.370348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1531.732212] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1539.209264] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1546.862522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1547.128962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1552.246301] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1552.308348] Lustre: 63698:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1552.333509] Lustre: 63698:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1552.346275] Lustre: 63698:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1552.357814] Lustre: 63698:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1552.373704] Lustre: 63698:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1552.387073] Lustre: 63698:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1553.568862] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1562.073378] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1566.888612] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1577.956498] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1577.963922] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1577.972398] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1581.882759] Lustre: server umount lustre-OST0000 complete [ 1585.214444] Lustre: server umount lustre-MDT0000 complete [ 1593.974500] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 1596.550445] Key type lgssc unregistered [ 1596.798237] LNet: 65259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1596.810048] LNetError: 65259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1596.826872] LNet: Removed LNI 192.168.204.153@tcp [ 1597.598228] Key type .llcrypt unregistered [ 1597.601623] Key type ._llcrypt unregistered [ 1614.238690] Key type ._llcrypt registered [ 1614.244564] Key type .llcrypt registered [ 1614.317675] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 1628.764662] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1629.591571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1629.709042] alg: No test for adler32 (adler32-zlib) [ 1630.739727] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 1630.954098] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 1632.639672] Key type lgssc registered [ 1634.035825] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1667.577613] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1667.628293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1668.848750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1668.893334] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1668.994378] Lustre: lustre-MDT0000: new disk, initializing [ 1669.103983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1669.123541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1673.349294] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1683.856124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1683.978848] Lustre: 68409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1684.064136] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1684.067624] Lustre: Skipped 1 previous similar message [ 1684.124255] Lustre: lustre-MDT0001: new disk, initializing [ 1684.211083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1684.248244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1684.259083] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1688.203851] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1697.549525] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1703.386265] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1709.406858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1709.689457] Lustre: lustre-OST0000: new disk, initializing [ 1709.695798] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1709.707628] Lustre: 70379:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1709.803380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1714.982432] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1715.777421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1715.785323] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1715.908448] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1723.645789] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1728.583515] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1728.860324] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1732.922831] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1733.094575] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1736.672754] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1736.674075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1736.684718] LustreError: Skipped 1 previous similar message [ 1736.687374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1736.727649] Lustre: Skipped 1 previous similar message [ 1741.142288] Lustre: server umount lustre-OST0000 complete [ 1745.909060] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1745.915150] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1745.929608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1745.934044] Lustre: Skipped 1 previous similar message [ 1751.707130] Lustre: server umount lustre-MDT0000 complete [ 1755.564071] LustreError: 68403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788273187 with bad export cookie 7921922015893368771 [ 1755.567919] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1755.578054] LustreError: 68403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1755.964185] Lustre: server umount lustre-MDT0001 complete [ 1765.191223] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:33:15 (1788273195) [ 1776.895721] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1813.062586] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 1823.846132] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 1856.507604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1856.804700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1856.857934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1856.928444] Lustre: lustre-MDT0000: new disk, initializing [ 1856.992572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1857.023039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1860.428784] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1869.285739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1869.378727] Lustre: 76397:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1869.454088] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1869.461919] Lustre: Skipped 1 previous similar message [ 1869.588667] Lustre: lustre-MDT0001: new disk, initializing [ 1869.682290] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1869.712715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1869.728168] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1874.095649] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1883.333223] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1889.342234] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1895.911340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1896.132919] Lustre: lustre-OST0000: new disk, initializing [ 1896.136363] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1896.152510] Lustre: 78365:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1896.231838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1901.651306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1901.664926] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1901.718380] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1901.776737] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1911.743770] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1917.188783] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1917.395849] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1922.947761] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1923.189813] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1927.653414] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1927.660194] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1927.662195] LustreError: Skipped 1 previous similar message [ 1927.666172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1927.687327] Lustre: Skipped 2 previous similar messages [ 1927.718109] Lustre: Skipped 3 previous similar messages [ 1931.471650] Lustre: server umount lustre-OST0000 complete [ 1936.374471] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1936.389342] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1936.407614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1943.009419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1943.028356] Lustre: Skipped 3 previous similar messages [ 1950.175203] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1950.432046] Lustre: server umount lustre-MDT0000 complete [ 1953.252759] LustreError: 76403:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1953.278458] LustreError: 76403:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 1954.513031] LustreError: 76388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788273385 with bad export cookie 7921922015893369646 [ 1954.517523] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1954.522931] LustreError: 76388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1954.897170] Lustre: server umount lustre-MDT0001 complete [ 1964.218481] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:36:34 (1788273394) [ 1971.251273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1971.717118] LustreError: 80779:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1971.785416] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1976.091312] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1976.806696] LustreError: 80780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1981.923692] LustreError: 80779:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1984.780573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1989.491691] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 1996.429468] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2001.374892] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2008.124315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2008.463252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2008.474523] Lustre: Skipped 1 previous similar message [ 2014.112640] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2021.774684] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2024.943269] Lustre: 83452:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 49e01079-56b8-46d3-abee-3014ef53da6d at adminstrative request [ 2026.984322] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2026.995558] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2037.219157] Lustre: MGS: Client 49e01079-56b8-46d3-abee-3014ef53da6d (at 0@lo) reconnecting [ 2037.236918] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x6df05219efabb6a7 to 0x6df05219efabb947 [ 2037.262469] Lustre: MGC192.168.204.153@tcp: Connection restored to 0@lo (at 0@lo) [ 2061.294465] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2061.464772] Lustre: lustre-OST0001: new disk, initializing [ 2061.469952] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2061.475183] Lustre: 83855:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2067.063181] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2067.087084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2067.098994] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2067.247597] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2076.355298] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2082.785440] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2082.792703] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2082.792716] Lustre: Skipped 1 previous similar message [ 2082.793868] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2082.810617] LustreError: Skipped 1 previous similar message [ 2082.826862] Lustre: Skipped 3 previous similar messages [ 2085.360670] Lustre: server umount lustre-OST0001 complete [ 2087.910284] LustreError: 83464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2087.927638] LustreError: 83464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2092.514035] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2092.520260] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2092.537757] Lustre: Skipped 1 previous similar message [ 2098.145559] LustreError: 82599:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2098.172069] LustreError: 82599:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2103.263989] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2103.267187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2103.280699] Lustre: Skipped 6 previous similar messages [ 2103.412794] Lustre: server umount lustre-OST0000 complete [ 2113.228614] Lustre: server umount lustre-MDT0000 complete [ 2116.437260] LustreError: 81518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788273547 with bad export cookie 7921922015893371207 [ 2116.437767] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2116.445282] LustreError: 81518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2116.716707] Lustre: server umount lustre-MDT0001 complete [ 2123.647469] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:39:13 (1788273553) [ 2124.994939] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2126.513348] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:39:16 (1788273556) [ 2127.781721] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2128.975248] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:39:19 (1788273559) [ 2130.263214] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2131.615092] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:39:22 (1788273562) [ 2133.211351] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2134.975106] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:39:25 (1788273565) [ 2136.530184] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2142.068714] loop: module loaded [ 2142.084862] loop0: detected capacity change from 0 to 4294967296 [ 2142.168600] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2143.383454] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2146.389751] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2150.216603] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2178.422511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2178.464815] LustreError: 87976:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2178.471974] LustreError: 87976:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2178.479695] LustreError: 87976:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2178.486684] LustreError: 87976:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2195.041265] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 2204.257929] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 2236.708839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2236.964863] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2237.020183] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2237.084331] Lustre: lustre-MDT0000: new disk, initializing [ 2237.167982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2237.175690] Lustre: Skipped 1 previous similar message [ 2237.197316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2241.563299] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2251.849964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2251.986396] Lustre: 90841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2252.323886] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2256.376931] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2265.372789] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2270.533590] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2276.347310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2276.609222] Lustre: 92809:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2282.005808] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2286.153301] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2286.318399] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2290.458585] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2293.861429] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2294.026532] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2297.520581] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2297.704763] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2301.921856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2301.924339] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2301.933202] Lustre: Skipped 3 previous similar messages [ 2301.943610] LustreError: Skipped 1 previous similar message [ 2301.946904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2301.946911] Lustre: Skipped 3 previous similar messages [ 2305.322239] Lustre: server umount lustre-OST0000 complete [ 2315.081548] Lustre: server umount lustre-MDT0000 complete [ 2317.281707] LustreError: 90852:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2317.296969] LustreError: 90852:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 2318.053521] LustreError: 90832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788273749 with bad export cookie 7921922015893371823 [ 2318.059124] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2318.063326] LustreError: 90832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2318.354394] Lustre: server umount lustre-MDT0001 complete [ 2326.294811] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:42:36 (1788273756) [ 2333.398258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2338.290967] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2346.761233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2351.290292] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2357.794521] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2363.838074] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2370.820719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2371.044100] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2371.053840] Lustre: Skipped 4 previous similar messages [ 2377.236429] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2384.078512] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2388.010813] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2388.207834] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2391.975862] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2392.192256] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2398.116185] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2398.265351] Lustre: lustre-OST0001: new disk, initializing [ 2398.268708] Lustre: Skipped 2 previous similar messages [ 2398.272336] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2398.275964] Lustre: Skipped 3 previous similar messages [ 2398.280447] Lustre: 98591:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2403.993726] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2405.990605] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2406.001992] Lustre: Skipped 2 previous similar messages [ 2406.006417] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2406.101076] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2411.362633] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2414.336694] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2414.482173] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2417.208057] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2417.339547] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2434.972572] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2458.819465] Lustre: Failing over lustre-MDT0000 [ 2459.213986] Lustre: server umount lustre-MDT0000 complete [ 2459.618624] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2459.631599] LustreError: Skipped 1 previous similar message [ 2459.641831] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2459.659500] Lustre: Skipped 2 previous similar messages [ 2459.664919] LustreError: 97425:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2459.683109] LustreError: 97425:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2478.544405] Lustre: 66636:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788273894/real 1788273894] req@00000000a58c7da5 x1875140214526464/t0(0) o400->MGC192.168.204.153@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788273910 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2478.575052] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2480.692277] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2480.697453] LDISKFS-fs (dm-0): recovery complete [ 2480.706545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2488.812895] LustreError: 66634:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000aefb8a78 x1875140214534272/t0(0) o250->MGC192.168.204.153@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2489.169888] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2492.643363] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2493.194392] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2493.195657] Lustre: 95217:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2494.445761] Lustre: 95221:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2494.458559] Lustre: 95221:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2494.485733] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2494.558807] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2494.887128] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2495.211911] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2495.221061] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2495.543099] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2495.555144] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2495.568229] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2495.576710] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2496.224381] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2496.235799] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2496.567649] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2496.581081] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2497.854106] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2497.870704] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2498.508342] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2498.517321] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2498.839131] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2498.846300] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2502.000255] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2502.014941] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2502.656810] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2502.668077] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2502.991280] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2502.995768] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2510.296525] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2510.308985] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2510.944104] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2510.953228] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2511.271813] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2511.283752] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2526.474963] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2526.485459] Lustre: 101442:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2527.129635] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2527.138273] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2527.463096] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2527.469815] LustreError: 101442:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2553.048131] Lustre: 101442:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2553.062477] Lustre: 101442:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2553.075980] Lustre: 101442:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1e8ae63a-6b12-45a4-aa29-cb9610cf02f0@192.168.204.53@tcp [ 2553.086180] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2553.091476] Lustre: 101442:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000516a6c61 x1875140199651456/t0(8589934781) o101->1e8ae63a-6b12-45a4-aa29-cb9610cf02f0@192.168.204.53@tcp:625/0 lens 592/0 e 0 to 0 dl 1788273995 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2553.115980] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2553.129983] Lustre: lustre-MDT0000: Denying connection for new client 1e8ae63a-6b12-45a4-aa29-cb9610cf02f0 (at 192.168.204.53@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2553.145991] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2553.166972] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2553.170522] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2553.174564] Lustre: Skipped 2 previous similar messages [ 2553.202698] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2553.208478] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2566.632899] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2566.641469] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2566.650025] Lustre: Skipped 3 previous similar messages [ 2566.653977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2566.656804] Lustre: Skipped 3 previous similar messages [ 2571.341733] Lustre: server umount lustre-OST0000 complete [ 2574.314446] LustreError: 97042:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2574.337178] LustreError: 97042:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 39 previous similar messages [ 2580.661201] Lustre: server umount lustre-OST0001 complete [ 2598.367160] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2598.501488] Lustre: server umount lustre-MDT0000 complete [ 2601.139403] LustreError: 95201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274032 with bad export cookie 7921922015893423042 [ 2601.145592] LustreError: 95201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2601.145613] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2601.420314] Lustre: server umount lustre-MDT0001 complete [ 2608.183522] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:47:18 (1788274038) [ 2613.505461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2613.511401] Lustre: *** cfs_fail_loc=197, val=0*** [ 2629.024532] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2634.144390] LustreError: 103413:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.153@tcp: failed processing log, type 4: rc = -110 [ 2639.286379] Lustre: *** cfs_fail_loc=197, val=0*** [ 2639.289355] Lustre: Skipped 65 previous similar messages [ 2659.871602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2659.877323] Lustre: Skipped 2 previous similar messages [ 2664.815973] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2670.217941] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2671.484929] Lustre: server umount lustre-OST0000 complete [ 2677.444439] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:48:27 (1788274107) [ 2703.996053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2705.403751] LustreError: 105033:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2705.458104] LustreError: 105033:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2715.442831] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2729.753859] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2734.957560] LustreError: 105064:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2734.983698] LustreError: 105064:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2734.997237] LustreError: 105064:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 29, retries 0, failed: rc = -5 [ 2735.455707] Lustre: server umount lustre-MDT0000 complete [ 2772.470937] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 2782.396993] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 2818.459044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.715249] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2818.764890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2818.884433] Lustre: lustre-MDT0000: new disk, initializing [ 2819.013302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2823.868671] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2837.158839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2837.384095] Lustre: 109242:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2837.824513] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2843.551252] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2857.167028] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2865.816714] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2874.273708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2874.656987] Lustre: 111211:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2881.838655] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2882.069815] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 2893.083923] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2898.291569] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2898.582545] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2903.790742] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2903.968704] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2907.618364] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2907.623427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2907.630364] LustreError: Skipped 4 previous similar messages [ 2907.632973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2907.632981] Lustre: Skipped 11 previous similar messages [ 2907.661193] Lustre: Skipped 6 previous similar messages [ 2912.304644] Lustre: server umount lustre-OST0000 complete [ 2922.982702] Lustre: server umount lustre-MDT0000 complete [ 2927.230480] LustreError: 112549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274358 with bad export cookie 7921922015893433738 [ 2927.237764] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2927.250494] LustreError: 112549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2927.624953] Lustre: server umount lustre-MDT0001 complete [ 2937.151307] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:52:46 (1788274366) [ 2951.057632] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 2954.543498] Key type lgssc unregistered [ 2954.879614] LNet: 113702:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2954.886208] LNetError: 113702:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2954.909141] LNet: Removed LNI 192.168.204.153@tcp [ 2956.101444] Key type .llcrypt unregistered [ 2956.104324] Key type ._llcrypt unregistered [ 2978.081333] Key type ._llcrypt registered [ 2978.087685] Key type .llcrypt registered [ 2978.219632] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 2997.760268] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 2999.147276] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2999.313275] alg: No test for adler32 (adler32-zlib) [ 3000.480605] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 3000.800583] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 3002.512094] Key type lgssc registered [ 3004.020605] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3054.355288] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3054.375934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3055.887497] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3055.960220] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3056.062406] Lustre: lustre-MDT0000: new disk, initializing [ 3056.200183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3056.229837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3064.666106] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3081.373060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3082.010658] Lustre: lustre-OST0000: new disk, initializing [ 3082.024749] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3082.033674] Lustre: Skipped 1 previous similar message [ 3082.049166] Lustre: 117881:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3082.151397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3089.704763] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3090.634819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3090.652229] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3090.690173] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3102.962522] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3103.126380] Lustre: lustre-OST0001: new disk, initializing [ 3103.129849] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3103.134888] Lustre: 118745:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3103.236842] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3110.515688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3110.523798] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3110.565178] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3110.566621] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3125.738769] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3125.746819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3125.751178] Lustre: Skipped 1 previous similar message [ 3127.806575] Lustre: server umount lustre-MDT0000 complete [ 3135.773038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3141.292118] LustreError: 117065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274572 with bad export cookie 530569090468079251 [ 3141.296654] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3141.305786] LustreError: 117065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3141.379724] Lustre: server umount lustre-OST0000 complete [ 3145.496606] Lustre: server umount lustre-OST0001 complete [ 3155.107193] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:56:24 (1788274584) [ 3183.671929] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:56:53 (1788274613) [ 3197.689848] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 3235.483268] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:57:44 (1788274664) [ 3297.890512] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 3314.494217] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 3359.232381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3359.613501] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3359.660346] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3359.847210] Lustre: lustre-MDT0000: new disk, initializing [ 3360.005943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3360.045629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3365.407808] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3379.759618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3380.018336] Lustre: 126529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3380.228927] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3380.242502] Lustre: Skipped 1 previous similar message [ 3380.507594] Lustre: lustre-MDT0001: new disk, initializing [ 3380.696864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3380.727544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3380.738403] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3387.050984] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3398.394822] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3405.391559] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3414.424980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3415.015062] Lustre: lustre-OST0000: new disk, initializing [ 3415.026386] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3415.039166] Lustre: 128500:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3415.218710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3421.269664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3421.292831] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3421.396151] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3423.207718] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3433.961681] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3439.753577] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3440.016589] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3445.662495] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3445.884529] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3452.389677] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3452.396606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3452.406787] LustreError: Skipped 1 previous similar message [ 3452.409184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3452.409191] Lustre: Skipped 1 previous similar message [ 3452.444716] Lustre: Skipped 1 previous similar message [ 3454.171325] Lustre: server umount lustre-OST0000 complete [ 3462.626699] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3462.630741] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3462.694861] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3462.720195] Lustre: Skipped 2 previous similar messages [ 3467.746616] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3467.755167] Lustre: Skipped 1 previous similar message [ 3472.870915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3472.879319] Lustre: Skipped 1 previous similar message [ 3474.399255] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3474.598799] Lustre: server umount lustre-MDT0000 complete [ 3478.971994] LustreError: 127333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788274910 with bad export cookie 530569090468080476 [ 3478.976974] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3478.991442] LustreError: 127333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3479.445111] Lustre: server umount lustre-MDT0001 complete [ 3489.264582] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:01:59 (1788274919) [ 3497.493943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3498.172782] LustreError: 130925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3498.206069] LustreError: 130925:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3498.276588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3503.590245] LustreError: 130926:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3504.668173] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3507.691551] LustreError: 130925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3512.809888] LustreError: 130925:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3515.004474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3521.368100] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3530.839379] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3538.568462] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3546.576379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3546.968104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3546.981686] Lustre: Skipped 1 previous similar message [ 3553.755629] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3562.691972] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3579.100436] Lustre: *** cfs_fail_loc=159, val=0*** [ 3594.582595] Lustre: lustre-MDT0000: Client 706e912c-e9ee-4848-b56a-1e48cd6b3915 (at 192.168.204.53@tcp) reconnecting [ 3596.700530] Lustre: *** cfs_fail_loc=159, val=0*** [ 3596.704144] Lustre: Skipped 7 previous similar messages [ 3612.982583] Lustre: lustre-MDT0000: Client 706e912c-e9ee-4848-b56a-1e48cd6b3915 (at 192.168.204.53@tcp) reconnecting [ 3631.078156] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3631.088240] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3631.093117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3631.106019] Lustre: Skipped 2 previous similar messages [ 3636.908579] Lustre: server umount lustre-OST0000 complete [ 3643.873336] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3643.885172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3649.020397] Lustre: server umount lustre-MDT0000 complete [ 3651.553201] LustreError: 133538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3651.563520] LustreError: 133538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3653.046773] LustreError: 133667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275084 with bad export cookie 530569090468081358 [ 3653.048277] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3653.053760] LustreError: 133667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3653.461985] Lustre: server umount lustre-MDT0001 complete [ 3668.066634] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 3670.686815] Key type lgssc unregistered [ 3671.085712] LNet: 134727:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3671.099343] LNetError: 134727:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3671.131056] LNet: Removed LNI 192.168.204.153@tcp [ 3672.713110] Key type .llcrypt unregistered [ 3672.714708] Key type ._llcrypt unregistered [ 3707.990315] Key type ._llcrypt registered [ 3707.994193] Key type .llcrypt registered [ 3708.141831] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 3709.287038] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3709.417845] alg: No test for adler32 (adler32-zlib) [ 3710.495332] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 3710.900996] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 3712.584068] Key type lgssc registered [ 3714.033238] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3724.842305] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3724.878780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3726.660872] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3732.866840] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3737.066095] LustreError: 136145:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3737.095451] LustreError: 136145:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3742.176228] LustreError: 136146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3744.726879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3745.242569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3750.531954] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3764.599915] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3775.931854] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3786.614439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3787.091501] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3795.542403] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 3797.430974] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3803.633361] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3814.892340] Lustre: *** cfs_fail_loc=159, val=0*** [ 3831.131528] Lustre: lustre-MDT0000: Client 1fe1e131-61f7-4cdf-971e-becf066ce74a (at 192.168.204.53@tcp) reconnecting [ 3835.692180] Lustre: *** cfs_fail_loc=159, val=0*** [ 3852.159034] Lustre: lustre-MDT0001: Client 1fe1e131-61f7-4cdf-971e-becf066ce74a (at 192.168.204.53@tcp) reconnecting [ 3854.284869] Lustre: *** cfs_fail_loc=159, val=0*** [ 3854.290159] Lustre: Skipped 3 previous similar messages [ 3869.552832] Lustre: lustre-MDT0001: Client 1fe1e131-61f7-4cdf-971e-becf066ce74a (at 192.168.204.53@tcp) reconnecting [ 3874.775508] Lustre: 139189:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3878.384129] Lustre: *** cfs_fail_loc=159, val=0*** [ 3878.386029] Lustre: Skipped 4 previous similar messages [ 3893.583626] Lustre: lustre-MDT0000: Client 66024f31-9ead-474c-a1cf-72e438efde9c (at 192.168.204.53@tcp) reconnecting [ 3896.010521] Lustre: *** cfs_fail_loc=159, val=0*** [ 3896.016524] Lustre: Skipped 14 previous similar messages [ 3911.505317] Lustre: lustre-MDT0000: Client 66024f31-9ead-474c-a1cf-72e438efde9c (at 192.168.204.53@tcp) reconnecting [ 3914.494365] Lustre: 139484:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3923.427654] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3923.447628] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3923.460968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3923.930947] Lustre: server umount lustre-OST0000 complete [ 3929.572371] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3929.580636] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3929.596295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3930.592566] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3930.618560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3930.631365] Lustre: Skipped 1 previous similar message [ 3935.714054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3935.723157] Lustre: Skipped 1 previous similar message [ 3938.896446] Lustre: server umount lustre-MDT0000 complete [ 3939.813207] LustreError: 136146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3939.840254] LustreError: 136146:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3943.732953] LustreError: 136127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275375 with bad export cookie 12965515965434528176 [ 3943.753366] LustreError: 136127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3943.762963] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3944.133954] Lustre: server umount lustre-MDT0001 complete [ 3956.269882] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 3959.401666] Key type lgssc unregistered [ 3959.712699] LNet: 140541:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3959.720830] LNetError: 140541:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3959.736749] LNet: Removed LNI 192.168.204.153@tcp [ 3960.867591] Key type .llcrypt unregistered [ 3960.869236] Key type ._llcrypt unregistered [ 3988.691955] Key type ._llcrypt registered [ 3988.694290] Key type .llcrypt registered [ 3988.807681] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 3989.933759] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3989.961288] alg: No test for adler32 (adler32-zlib) [ 3991.004546] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 3991.209455] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 3992.887165] Key type lgssc registered [ 3994.264845] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4002.064414] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4002.100996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4003.670814] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4008.007349] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4014.062261] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4014.081452] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4016.768700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4017.131181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4021.589983] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4029.949424] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4036.134216] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4046.611288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4047.329421] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4049.445577] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4055.476328] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4061.759756] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4065.212530] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4072.819454] Lustre: 144542:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4076.322619] Lustre: 144591:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4077.791410] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4078.837613] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4083.173724] Lustre: 144737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4084.018559] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4085.212261] Lustre: 144835:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4090.337781] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4090.345469] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4090.359983] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4092.387733] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4092.397071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4092.399123] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4092.450520] Lustre: Skipped 1 previous similar message [ 4096.563167] Lustre: server umount lustre-OST0000 complete [ 4102.628309] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4102.637451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4102.644221] Lustre: Skipped 1 previous similar message [ 4102.656790] Lustre: Skipped 1 previous similar message [ 4107.745312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4107.753466] Lustre: Skipped 1 previous similar message [ 4111.843665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4114.403221] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4114.570307] Lustre: server umount lustre-MDT0000 complete [ 4116.964051] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4116.987343] LustreError: 141956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4118.668504] LustreError: 142690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275550 with bad export cookie 10108070052171802174 [ 4118.668504] LustreError: 141937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275550 with bad export cookie 10108070052171802174 [ 4118.671131] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4118.675196] LustreError: 141937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4119.069529] Lustre: server umount lustre-MDT0001 complete [ 4129.201749] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 4131.910694] Key type lgssc unregistered [ 4132.264717] LNet: 145900:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4132.273870] LNetError: 145900:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4132.297612] LNet: Removed LNI 192.168.204.153@tcp [ 4133.263625] Key type .llcrypt unregistered [ 4133.267463] Key type ._llcrypt unregistered [ 4157.872539] Key type ._llcrypt registered [ 4157.876041] Key type .llcrypt registered [ 4157.984204] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 4158.905798] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4159.007259] alg: No test for adler32 (adler32-zlib) [ 4160.309438] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 4160.866987] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 4162.760187] Key type lgssc registered [ 4164.878112] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4175.244967] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4175.278183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4176.712497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4182.063797] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4187.114256] LustreError: 147316:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4187.144506] LustreError: 147316:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4192.135787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4192.225540] LustreError: 147317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4192.712177] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4198.081879] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4208.121500] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4215.564224] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4227.391671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4228.396505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4237.295463] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4237.620866] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4246.862905] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4248.028544] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4256.322513] Lustre: *** cfs_fail_loc=159, val=0*** [ 4256.336554] Lustre: Skipped 2 previous similar messages [ 4271.950720] Lustre: lustre-MDT0000: Client 502fceb6-ca0d-462f-8a61-fcaab8a4c24b (at 192.168.204.53@tcp) reconnecting [ 4293.098201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4293.101178] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4293.105589] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4293.122699] Lustre: Skipped 1 previous similar message [ 4293.152876] Lustre: Skipped 1 previous similar message [ 4298.233623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4298.250672] Lustre: Skipped 1 previous similar message [ 4303.332628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4303.351790] Lustre: Skipped 1 previous similar message [ 4304.351155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4304.665294] Lustre: server umount lustre-OST0000 complete [ 4310.496623] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4310.502101] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4310.521924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4313.596222] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4316.004239] Lustre: server umount lustre-MDT0000 complete [ 4318.708640] LustreError: 148291:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4318.774152] LustreError: 148291:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4321.268549] LustreError: 147296:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275752 with bad export cookie 4853780672654072262 [ 4321.280280] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4321.986249] Lustre: server umount lustre-MDT0001 complete [ 4338.427524] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 4342.520629] Key type lgssc unregistered [ 4342.901830] LNet: 150993:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4342.910405] LNetError: 150993:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4342.993629] LNet: Removed LNI 192.168.204.153@tcp [ 4344.474317] Key type .llcrypt unregistered [ 4344.477096] Key type ._llcrypt unregistered [ 4379.288758] Key type ._llcrypt registered [ 4379.297734] Key type .llcrypt registered [ 4379.515617] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 4381.782292] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4381.868515] alg: No test for adler32 (adler32-zlib) [ 4383.787135] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 4384.444859] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 4386.415360] Key type lgssc registered [ 4389.398759] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4403.028429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4403.058978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4404.968502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4409.679595] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4415.457364] LustreError: 152410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4415.475269] LustreError: 152410:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4419.483113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4419.906300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4424.773831] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4433.714567] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4440.804371] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4448.832037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4449.344224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4450.371700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4457.264121] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4464.692502] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4469.430474] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4490.601274] Lustre: lustre-OST0000: evicting cc7f1dc9-3942-40d6-a3f2-e284217d5298 (at 192.168.204.53@tcp) by administrative request [ 4490.611915] Lustre: Skipped 1 previous similar message [ 4520.929484] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4520.934697] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4520.936633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4520.941885] Lustre: Skipped 1 previous similar message [ 4525.968851] Lustre: server umount lustre-OST0000 complete [ 4532.709791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4532.718304] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.736898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4532.744927] Lustre: Skipped 1 previous similar message [ 4536.292100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4536.294039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4536.327848] Lustre: Skipped 1 previous similar message [ 4537.874160] Lustre: server umount lustre-MDT0000 complete [ 4541.413327] LustreError: 152404:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4541.443982] LustreError: 152404:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4542.434668] LustreError: 152410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4542.463772] LustreError: 152410:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4542.828882] LustreError: 152389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788275974 with bad export cookie 1842257419483896675 [ 4542.833875] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4542.845479] LustreError: 152389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4543.138119] Lustre: server umount lustre-MDT0001 complete [ 4556.081947] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 4559.739423] Key type lgssc unregistered [ 4560.139149] LNet: 156482:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4560.147235] LNetError: 156482:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4560.167105] LNet: Removed LNI 192.168.204.153@tcp [ 4561.290654] Key type .llcrypt unregistered [ 4561.292668] Key type ._llcrypt unregistered [ 4655.284806] Key type ._llcrypt registered [ 4655.288212] Key type .llcrypt registered [ 4655.433520] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 4656.703705] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4656.767629] alg: No test for adler32 (adler32-zlib) [ 4657.857872] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 4658.144854] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 4659.855323] Key type lgssc registered [ 4661.145403] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4669.806474] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4669.833250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4671.397738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4675.804947] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4680.707176] LustreError: 158974:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4680.736494] LustreError: 158974:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4684.936303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4685.450032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4690.281885] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4699.777865] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4706.236730] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4713.766578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4714.245733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4716.279224] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4721.046488] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4729.008612] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4729.593086] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4740.070953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4740.073268] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4740.080240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4740.087451] Lustre: Skipped 1 previous similar message [ 4745.186235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4745.196433] Lustre: Skipped 1 previous similar message [ 4745.421826] Lustre: server umount lustre-OST0000 complete [ 4750.315262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4750.334717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4750.344466] Lustre: Skipped 2 previous similar messages [ 4755.428237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4755.443282] Lustre: Skipped 1 previous similar message [ 4756.180885] Lustre: server umount lustre-MDT0000 complete [ 4760.432650] LustreError: 158955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276191 with bad export cookie 3884366638583095009 [ 4760.437383] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4760.450981] LustreError: 158955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4760.910259] Lustre: server umount lustre-MDT0001 complete [ 4774.671657] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 4777.630554] Key type lgssc unregistered [ 4777.975296] LNet: 162519:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4777.986963] LNetError: 162519:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4779.054774] LNet: Removed LNI 192.168.204.153@tcp [ 4779.983927] Key type .llcrypt unregistered [ 4779.986398] Key type ._llcrypt unregistered [ 4810.552994] Key type ._llcrypt registered [ 4810.555342] Key type .llcrypt registered [ 4810.655087] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 4811.990049] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4812.035138] alg: No test for adler32 (adler32-zlib) [ 4813.217749] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 4813.573555] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 4815.295510] Key type lgssc registered [ 4816.912395] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4833.842860] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:24:23 (1788276263) [ 4853.449296] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 4864.458167] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 4906.098245] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4906.123524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4907.464564] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4907.488766] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4907.582862] Lustre: lustre-MDT0000: new disk, initializing [ 4907.706769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4907.741854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4913.389355] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4926.218694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4926.380342] Lustre: 166786:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4926.451535] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4926.456554] Lustre: Skipped 1 previous similar message [ 4926.552812] Lustre: lustre-MDT0001: new disk, initializing [ 4926.710634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4926.789706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4926.808733] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4932.206295] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4944.435571] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4951.075739] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4959.232532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4959.600546] Lustre: lustre-OST0000: new disk, initializing [ 4959.604634] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4959.610425] Lustre: 168757:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4959.695132] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4967.238593] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 4969.069634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4969.078794] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4969.190033] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4977.442587] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4989.927389] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4989.930339] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4989.931784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4989.943438] Lustre: Skipped 1 previous similar message [ 4989.984220] LustreError: Skipped 1 previous similar message [ 4995.041142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4995.061589] Lustre: Skipped 2 previous similar messages [ 4995.576154] Lustre: server umount lustre-OST0000 complete [ 5000.160800] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5000.191568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5000.200098] Lustre: Skipped 1 previous similar message [ 5005.282587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5005.292494] Lustre: Skipped 1 previous similar message [ 5006.315707] Lustre: server umount lustre-MDT0000 complete [ 5010.918689] LustreError: 167583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788276442 with bad export cookie 4153213708918164743 [ 5010.919181] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5010.944992] LustreError: 167583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5011.638303] Lustre: server umount lustre-MDT0001 complete [ 5026.737606] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 5030.359434] Key type lgssc unregistered [ 5030.663383] LNet: 170665:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5030.682926] LNetError: 170665:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5030.728905] LNet: Removed LNI 192.168.204.153@tcp [ 5031.814970] Key type .llcrypt unregistered [ 5031.816995] Key type ._llcrypt unregistered [ 5060.926905] Key type ._llcrypt registered [ 5060.929361] Key type .llcrypt registered [ 5061.056294] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 5062.162622] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5062.355090] alg: No test for adler32 (adler32-zlib) [ 5063.554080] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 5063.854583] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 5065.559162] Key type lgssc registered [ 5067.028613] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5075.654564] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5075.674819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5077.408540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5082.753200] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 5086.691636] LustreError: 172080:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5086.716667] LustreError: 172080:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5091.819932] LustreError: 172081:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5093.904828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5094.494679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5099.600689] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 5108.665537] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5116.167504] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5124.575152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5124.890377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5133.173024] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 5143.862402] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5147.476538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5156.951603] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5156.977337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5157.004584] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5157.015216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.037324] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5158.316817] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5158.370433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5158.405759] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5158.445188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.483550] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5159.685869] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5159.735488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5159.764157] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5159.793552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.811550] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5161.260788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.874031] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5162.884109] Lustre: Skipped 1 previous similar message [ 5162.917919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5162.937551] Lustre: Skipped 1 previous similar message [ 5162.964905] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5162.978354] Lustre: Skipped 1 previous similar message [ 5162.983594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.002570] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5163.010666] Lustre: Skipped 1 previous similar message [ 5164.304131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.081929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.343853] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5167.352167] Lustre: Skipped 2 previous similar messages [ 5167.377952] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5167.385878] Lustre: Skipped 2 previous similar messages [ 5167.396142] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5167.398611] Lustre: Skipped 2 previous similar messages [ 5167.406192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.417423] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5167.420033] Lustre: Skipped 2 previous similar messages [ 5168.738949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.206846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.614810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.972196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.510566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.636976] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5175.650386] Lustre: Skipped 5 previous similar messages [ 5175.692771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5175.712048] Lustre: Skipped 5 previous similar messages [ 5175.718646] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5175.727466] Lustre: Skipped 5 previous similar messages [ 5175.732467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.753843] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5175.769766] Lustre: Skipped 5 previous similar messages [ 5177.196189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.588549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.415901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.735701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.909399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.100099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.035588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.157286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.358547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.395386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.061658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.558904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.663627] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5192.677217] Lustre: Skipped 12 previous similar messages [ 5192.713822] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5192.734573] Lustre: Skipped 12 previous similar messages [ 5192.739934] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5192.746489] Lustre: Skipped 12 previous similar messages [ 5192.751142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.789606] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5192.806236] Lustre: Skipped 12 previous similar messages [ 5193.983561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.277584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.242497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.438978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.441407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.777553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.854557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.644918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.782680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.905379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.765756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.806905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.666252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.520433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.400298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.292021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.352420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.284911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.407167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.595686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.745379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.608777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.890837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.292057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.543050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.599246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.527768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.510142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.909110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.176241] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5225.183257] Lustre: Skipped 29 previous similar messages [ 5225.199112] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5225.217671] Lustre: Skipped 29 previous similar messages [ 5225.221883] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5225.230155] Lustre: Skipped 29 previous similar messages [ 5225.240123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.256339] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5225.262401] Lustre: Skipped 29 previous similar messages [ 5226.282543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.190279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.988425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.019570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.000393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.129939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.196461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.155790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.323441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.253799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.405531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.413924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.566964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.581331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.527270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.322855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.440525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.348995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.395686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.559032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.876994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.853856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.804673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.836085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.979925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.933596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.973219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.819945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.839607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.781075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.650162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.462278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.371209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.306183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.274856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.195883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.164523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.154783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.075645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.039092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.111678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.298630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.184064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.178724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.098355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.943825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.815983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.699712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.619633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.767197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.757913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.900191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.939239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.179718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.964935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.980184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.864632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.673584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.692883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.546324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.488644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.366542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.681982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.626903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.522148] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5289.529627] Lustre: Skipped 64 previous similar messages [ 5289.551766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5289.561300] Lustre: Skipped 64 previous similar messages [ 5289.565648] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5289.569457] Lustre: Skipped 64 previous similar messages [ 5289.572707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.584389] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5289.590484] Lustre: Skipped 64 previous similar messages [ 5290.403188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.383825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.340957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.103732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.859648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.780024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.674804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.761742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.597084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.578129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.459393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.273883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.086425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.954675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.908461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.076769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.934100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.995061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.960972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.052254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.995897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.094592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.019305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.014117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.044522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.069899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.144470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.014728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.051375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.989021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.910078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.881322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.058065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.037037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.034891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.904435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.688240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.540932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.307083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.173174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.986883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.779862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.634266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.546581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.567993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.311164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.171955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.015318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.054556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.047704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.099095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.148131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.136125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.220648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.196719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.239140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.221282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.242411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.364369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.579649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.616952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.592326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.578751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.576966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.645355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.813057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.873285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.926614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.072883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.957582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.968109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.855539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.778291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.592277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.684211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.611544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.518971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.442727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.453042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.388065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.409348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.404311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.449317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.446801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.547927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.379897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.398283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.452675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.556536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.455948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.601520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.794849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.958612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.880899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.756090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.690223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.683643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.474784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.390911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.345630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.101069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.008462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.049887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.942648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.854666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.775175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.850904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.792740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.956358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.942728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.949198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.852629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.853336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.829466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.872728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.711652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.015358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.057891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.292604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.276050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.471460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.413908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.353916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.287155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.340446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.232970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.299508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.225959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.168209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.181305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.191985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.202820] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5418.208280] Lustre: Skipped 131 previous similar messages [ 5418.218961] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5418.229977] Lustre: Skipped 131 previous similar messages [ 5418.234096] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5418.244702] Lustre: Skipped 131 previous similar messages [ 5418.252881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.269608] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5418.276749] Lustre: Skipped 131 previous similar messages [ 5419.178563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.049341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.149792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.986817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.108694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.082392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.274506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.404455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.450904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.566758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.758520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.927419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.870806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.875272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.778823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.047203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.213035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.478903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.522358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.323438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.334224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.297990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.180544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.278853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.385423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.440933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.430543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.564319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.535721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.985016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.294465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.531491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.868162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.122762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.335537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.925633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.432157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.568599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.457392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.644815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.596887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.913349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.421188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.450176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.748817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.084256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.535990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.227606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.933509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.282402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.674068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.873916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.367741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.727950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.144149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.579860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.999597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.322167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.499040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.833650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.531047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.982038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.138468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.361277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.306131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.280438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.331226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.345609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.622413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.055225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.187834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.365948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.653312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.889070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.099756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.465122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.545500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.622974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.749084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.625326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.010408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.069950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.043418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.122703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.146130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.513945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.831682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.005124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.208896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.224103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.440495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.757090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.853915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.180431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.985583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.128028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.495177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.474833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.452483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.803521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.760497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.022642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.074689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.953530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.036332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.149775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.411959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.317813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.403762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.416135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.613874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.460288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.448195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.242891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.001496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.212547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.199519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.190094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.367251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.686939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.966887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.201777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.202863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.276874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.208686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.258988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.594747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.640988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.757171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.989509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.117197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.237423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.432729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.459453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.405195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.320000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.643681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.605681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.525958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.439606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.531118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.570503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.607494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.773156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.701491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.664492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.667805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.791437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.860398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.982581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.907858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.181773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.027867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.191445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.216499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.107477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.056039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.203490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.249765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.273388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.300399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.307887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.252528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.155617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.154653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.189923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.185842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.274102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.366135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.420916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.380075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.451575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.633184] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5613.641842] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5613.661529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.765790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.980750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.051131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.242974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.374091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.407623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.436090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.503551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.481157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.353841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.283479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.268339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.361794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.438187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.511726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.613385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.865477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.860562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.799590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.673173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.576680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.763985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.830300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.875982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.701975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.769247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.713634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.827947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.772691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.802294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.665376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.688787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.714268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.715942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.686152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.771671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.689252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.777793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.816696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.795214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.643061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.685364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.712568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.790140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.919395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.950137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.963439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.991303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.931761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.210809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.288636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.428505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.366488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.343024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.192507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.365785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.471880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.418549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.676270] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5674.685469] Lustre: Skipped 231 previous similar messages [ 5674.699453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5674.707747] Lustre: Skipped 231 previous similar messages [ 5674.712292] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5674.717880] Lustre: Skipped 231 previous similar messages [ 5674.722930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.735273] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5674.749151] Lustre: Skipped 231 previous similar messages [ 5675.756589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.783027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.892550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.960534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.095880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.296055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.441781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.674650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.649059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.710847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.619575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.557906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.632736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.703573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.628883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.763820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.730872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.598552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.612098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.662540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.724584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.702914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.934315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.100388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.339925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.404763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.415917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.702414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.710625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.627883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.564468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.582911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.880967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.900457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.991882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.017994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.894032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.089681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.194453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.272642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.222615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.044114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.044026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.956282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.984503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.875569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.784680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.182261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.164817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.418571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.348092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.475251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.564465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.544117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.516170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.351455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.158217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.142085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.225293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.260176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.409507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.520849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.462945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.342203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.848377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.943290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.560187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.746375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.828125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.985905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.102803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.143838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.216404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.321167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.268065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.164106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.031840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.055646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.978113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.270036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.454265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.667488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.776798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.952202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.884796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.962923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.907822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.904313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.818590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.839706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.821216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.760378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.714629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.662245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.851813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.907107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.877816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.857727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.769734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.679716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.446648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.248806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.071890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.028452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.999510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.858027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.682487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.807917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.750861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.652719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.530635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.571136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.379886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.293897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.196625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.138978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.030529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.015047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.997275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.931796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.875249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.901929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.904467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.922439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.902291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.879700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.753672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.584149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.363194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.416530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.104975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.952387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.739384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.780204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.779407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.657108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.538629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.691773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.896066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.982741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.152721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.159295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.497769] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5820.521159] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5820.537468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.691662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.843254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.281659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.479986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.681134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.866957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.124993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.226887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.252274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.222284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.296274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.392684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.474882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.778328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.870675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.586275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.808122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.857950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.931750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.741802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.863900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.308379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.146315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.281076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.025956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.025853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.973073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.296199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.536773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.205522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.415606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.422953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.710643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.004800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.722597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.951549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.185487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.729567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.169694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.290810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.217216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.420716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.306665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.430389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.795268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.976548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.238396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.382733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.617829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.946879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.159159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.191107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.359979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.245736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.285827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.254856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.502550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.519639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.489584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.418760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.238577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.154555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.232557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.142244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.958671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.774145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.827102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.778359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.652510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.602971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.685453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.625974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.655348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.550301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.544373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.338506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.267059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.167810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.060952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.950253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.855349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.817945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.670666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.460296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.335850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.060254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.875296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.651541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.507150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.388663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.247527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.147741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.894513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.850174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.780677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.608956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.516274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.558804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.620867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.582642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.408496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.192615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.040033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.818297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.758147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.750793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.564237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.404835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.275062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.140716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.073928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.868274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.695727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.494460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.411794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.205810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.206028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.096117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.053829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.824654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.636254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.528213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.546073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.519339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.391697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.223503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.162354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.079804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.033496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.887175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.768177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.591266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.589260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.576746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.665650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.512827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.473943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.295222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.334254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.290889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.186287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.143470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.962084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.824497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.750615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.624505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.666681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.684894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.668194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.742836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.906414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.128208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.093260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.123478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.123448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.293963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.352710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.308770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.365218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.304349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.550554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.597081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.855131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.941466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.901832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.876965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.888104] LustreError: 172100:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5988.896146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.900357] LustreError: 172100:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5989.876738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.857655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.033434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.095423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.293378] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6157.303333] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6157.312507] Lustre: Skipped 1 previous similar message [ 6160.968552] Lustre: server umount lustre-OST0000 complete [ 6167.541319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6167.554970] Lustre: Skipped 1 previous similar message [ 6172.177322] Lustre: server umount lustre-MDT0000 complete [ 6172.649942] LustreError: 172077:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6172.663033] LustreError: 172077:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6176.742653] LustreError: 180383:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6176.816629] LustreError: 180383:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6177.196175] LustreError: 172061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788277608 with bad export cookie 18276685853557485431 [ 6177.202098] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6177.204588] LustreError: 172061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6177.573466] Lustre: server umount lustre-MDT0001 complete [ 6196.359644] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 6198.819788] Key type lgssc unregistered [ 6199.061741] LNet: 217006:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6199.069363] LNetError: 217006:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6199.097173] LNet: Removed LNI 192.168.204.153@tcp [ 6200.448130] Key type .llcrypt unregistered [ 6200.452663] Key type ._llcrypt unregistered [ 6230.908694] Key type ._llcrypt registered [ 6230.910628] Key type .llcrypt registered [ 6231.026412] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 6232.830785] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6232.867095] alg: No test for adler32 (adler32-zlib) [ 6234.040838] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 6234.305940] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 6235.975329] Key type lgssc registered [ 6237.662294] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6250.670215] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6250.736070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6252.792382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6260.028522] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6263.268323] LustreError: 218428:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6263.299439] LustreError: 218428:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6268.388925] LustreError: 218427:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6271.197950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6271.488762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6276.263952] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6285.304823] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6291.136316] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6298.211681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6298.492610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6305.043695] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6313.043864] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6323.138222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14259 to 0x280000401:14305) [ 6327.680134] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6329.387371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6329.401499] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6329.408189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.421522] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6329.422121] LustreError: 218447:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6331.368170] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6331.382941] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6331.398721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6333.414980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6333.428719] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6337.508820] Lustre: server umount lustre-OST0000 complete [ 6349.503189] LustreError: 218447:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6373.104864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6373.465655] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6374.521709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.565770] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6382.522518] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6383.787062] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.825996] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6393.053968] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6395.586227] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6403.307720] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0cc4649f800.ost_server_uuid 50 [ 6405.321832] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0cc4649f800.ost_server_uuid in IDLE state after 0 sec [ 6409.696774] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6409.703065] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6409.722567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6409.727973] Lustre: Skipped 2 previous similar messages [ 6414.816698] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6414.841059] Lustre: Skipped 1 previous similar message [ 6415.797774] Lustre: server umount lustre-OST0000 complete [ 6425.056515] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6425.061301] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6425.067711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6425.067719] Lustre: Skipped 1 previous similar message [ 6425.103095] Lustre: Skipped 2 previous similar messages [ 6426.517296] Lustre: server umount lustre-MDT0000 complete [ 6430.184743] LustreError: 218427:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6430.211377] LustreError: 218427:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6430.482405] LustreError: 218407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788277861 with bad export cookie 7147929332454283138 [ 6430.493796] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6430.504088] LustreError: 218407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6430.823436] Lustre: server umount lustre-MDT0001 complete [ 6443.334249] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 6446.513433] Key type lgssc unregistered [ 6446.810694] LNet: 223662:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6446.818370] LNetError: 223662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6446.842745] LNet: Removed LNI 192.168.204.153@tcp [ 6447.667776] Key type .llcrypt unregistered [ 6447.670657] Key type ._llcrypt unregistered [ 6483.946150] Key type ._llcrypt registered [ 6483.952130] Key type .llcrypt registered [ 6484.166449] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 6485.416387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6485.487517] alg: No test for adler32 (adler32-zlib) [ 6486.636135] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 6486.989858] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 6488.823165] Key type lgssc registered [ 6490.655537] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6503.665949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6503.708760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6505.487866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6510.926435] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6514.657595] LustreError: 225394:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6514.682452] LustreError: 225394:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6519.784457] LustreError: 225393:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6521.023624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6521.350091] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6527.229642] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6530.899544] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6539.002558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6539.482386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6546.703782] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6549.777371] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14307 to 0x280000401:14337) [ 6557.217282] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6557.429587] Lustre: lustre-OST0001: new disk, initializing [ 6557.432790] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6557.438095] Lustre: 227706:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6557.569047] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6564.017199] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6564.030559] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6564.285782] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6567.580111] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6581.011461] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6588.731667] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6605.281183] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6605.290241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6605.302622] Lustre: Skipped 3 previous similar messages [ 6605.309226] Lustre: Skipped 2 previous similar messages [ 6609.945376] Lustre: server umount lustre-MDT0000 complete [ 6610.404521] LustreError: 225390:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6610.425531] LustreError: 225390:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 6613.834409] LustreError: 228534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278045 with bad export cookie 8543918592202251221 [ 6613.836821] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6613.855301] LustreError: 228534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6614.171681] Lustre: server umount lustre-MDT0001 complete [ 6629.581268] Lustre: server umount lustre-OST0000 complete [ 6631.391218] Lustre: 224655:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788278046/real 1788278046] req@ffff997992aeca80 x1875145306244480/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788278062 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6631.440840] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6634.976107] Lustre: 224656:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788278050/real 1788278050] req@ffff99798ac8b100 x1875145306244736/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788278066 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6635.626155] Lustre: server umount lustre-OST0001 complete [ 6662.572743] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 6675.882491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6676.836835] LustreError: 230815:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6676.929548] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6682.043754] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6682.082797] LustreError: 230816:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6692.263641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6692.334832] LustreError: 230815:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6692.374408] LustreError: 230815:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6693.022238] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6698.364699] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6701.702845] Lustre: 231956:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6710.295043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6710.900757] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6716.885457] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6718.053388] LustreError: 232309:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6722.183870] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14339 to 0x280000401:14369) [ 6726.004775] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6731.802064] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6733.133882] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6741.221599] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6745.671820] Lustre: 233829:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6759.394635] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6759.405743] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6759.441889] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6759.456129] Lustre: Skipped 1 previous similar message [ 6760.444411] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6760.468651] Lustre: Skipped 3 previous similar messages [ 6765.540073] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6765.552214] Lustre: Skipped 3 previous similar messages [ 6770.145863] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6770.444787] Lustre: server umount mylustre-MDT0000 complete [ 6770.682604] LustreError: 233595:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6770.720129] LustreError: 233595:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 6776.284137] LustreError: 230790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278207 with bad export cookie 8543918592202254028 [ 6776.305775] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6776.309803] LustreError: 230790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6777.001207] Lustre: server umount mylustre-MDT0001 complete [ 6792.390374] Lustre: server umount mylustre-OST0000 complete [ 6807.759501] Lustre: server umount mylustre-OST0001 complete [ 6832.541689] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 6843.144719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6843.529947] LustreError: 236141:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6843.550251] LustreError: 236141:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 6843.615649] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6843.621227] Lustre: Skipped 1 previous similar message [ 6848.203322] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6857.677480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6863.063832] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6866.450790] Lustre: 237285:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6873.451963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6881.007957] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6890.487140] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14339 to 0x280000401:14401) [ 6890.671831] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6896.118447] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 6897.692596] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 6906.162049] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6910.454178] Lustre: 239157:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6919.652604] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6919.667131] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6919.684256] Lustre: Skipped 3 previous similar messages [ 6919.691321] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6920.877708] Lustre: server umount tfs-MDT0000 complete [ 6924.213960] LustreError: 236118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278355 with bad export cookie 8543918592202255918 [ 6924.224876] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6924.226647] LustreError: 236118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6924.645929] Lustre: server umount tfs-MDT0001 complete [ 6937.809910] Lustre: server umount tfs-OST0000 complete [ 6951.937091] Lustre: server umount tfs-OST0001 complete [ 6976.246765] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 6987.244995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6987.717488] LustreError: 241409:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6987.734145] LustreError: 241409:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 6987.797343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6987.802310] Lustre: Skipped 3 previous similar messages [ 6993.201761] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7002.274766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7007.397360] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7010.827080] Lustre: 242552:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7017.334241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7023.284597] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7031.256166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7032.583219] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:14339 to 0x280000401:14433) [ 7032.584330] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7037.585403] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7045.700898] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7049.760777] Lustre: 244437:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7057.901953] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:01:27 (1788278487) [ 7074.272085] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7074.282058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7074.303047] Lustre: Skipped 3 previous similar messages [ 7074.317449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7080.702751] Lustre: server umount lustre-MDT0000 complete [ 7085.420549] LustreError: 241385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278516 with bad export cookie 8543918592202257584 [ 7085.433972] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7085.442120] LustreError: 241385:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7089.125146] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7089.149811] Lustre: Skipped 3 previous similar messages [ 7092.041429] Lustre: server umount lustre-MDT0001 complete [ 7111.135135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7111.311059] Lustre: server umount lustre-OST0000 complete [ 7121.717736] Lustre: server umount lustre-OST0001 complete [ 7129.808804] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 7142.081658] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 7180.440557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7180.669703] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7180.699197] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7180.804269] Lustre: lustre-MDT0000: new disk, initializing [ 7180.882960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7180.889414] Lustre: Skipped 3 previous similar messages [ 7180.901921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7185.691431] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7196.495440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7196.589096] Lustre: 247621:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7196.641267] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7196.644814] Lustre: Skipped 1 previous similar message [ 7196.711891] Lustre: lustre-MDT0001: new disk, initializing [ 7196.841641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7196.854670] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7201.705700] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7216.962535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7217.607472] Lustre: lustre-OST0000: new disk, initializing [ 7217.620856] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7217.645063] Lustre: 249203:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7219.393057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7219.415430] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7219.741875] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7229.876989] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7244.731926] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7244.935837] Lustre: lustre-OST0001: new disk, initializing [ 7244.941429] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7244.950722] Lustre: 250075:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7246.390485] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7246.410942] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7246.546493] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7253.626653] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7268.806277] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:04:58 (1788278698) [ 7284.260643] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:05:14 (1788278714) [ 7301.600752] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7301.602456] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7301.611065] LustreError: Skipped 1 previous similar message [ 7301.614027] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7301.614035] Lustre: Skipped 8 previous similar messages [ 7301.651222] Lustre: Skipped 2 previous similar messages [ 7314.400039] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7314.626173] Lustre: server umount lustre-OST0000 complete [ 7318.501865] LustreError: 249188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7318.512506] LustreError: 249188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7324.593846] Lustre: server umount lustre-MDT0000 complete [ 7328.550799] LustreError: 247615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788278760 with bad export cookie 8543918592202259131 [ 7328.561112] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7328.572828] LustreError: 247615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7333.856637] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7333.863426] Lustre: Skipped 14 previous similar messages [ 7335.083276] Lustre: server umount lustre-MDT0001 complete [ 7346.737436] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 7418.567954] Lustre: server umount lustre-OST0001 complete [ 7425.989132] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 7444.814061] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 7489.698845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7489.988541] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7490.024101] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7490.135346] Lustre: lustre-MDT0000: new disk, initializing [ 7490.226439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7490.232486] Lustre: Skipped 3 previous similar messages [ 7490.260141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7495.579241] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7506.393289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7506.557986] Lustre: 254929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7506.646661] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7506.653381] Lustre: Skipped 1 previous similar message [ 7506.763552] Lustre: lustre-MDT0001: new disk, initializing [ 7506.905908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7506.924101] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7511.433638] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7521.699318] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7527.256102] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7533.269801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7533.478648] Lustre: lustre-OST0000: new disk, initializing [ 7533.484112] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7533.491839] Lustre: 256896:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7534.744818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7534.758917] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7534.857541] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7540.044346] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7550.146100] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7560.678678] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7560.682207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7560.694677] LustreError: Skipped 1 previous similar message [ 7560.700649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7560.722353] Lustre: Skipped 5 previous similar messages [ 7560.735263] Lustre: Skipped 1 previous similar message [ 7564.699303] Lustre: server umount lustre-OST0000 complete [ 7583.199270] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7583.558665] Lustre: server umount lustre-MDT0000 complete [ 7587.409123] LustreError: 256906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279018 with bad export cookie 8543918592202260608 [ 7587.418628] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7587.423134] LustreError: 256906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7587.749810] Lustre: server umount lustre-MDT0001 complete [ 7600.024777] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 7602.828970] Key type lgssc unregistered [ 7603.188557] LNet: 258660:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7603.199669] LNetError: 258660:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7603.245662] LNet: Removed LNI 192.168.204.153@tcp [ 7604.199174] Key type .llcrypt unregistered [ 7604.202449] Key type ._llcrypt unregistered [ 7632.840784] Key type ._llcrypt registered [ 7632.842737] Key type .llcrypt registered [ 7632.964639] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 7634.056287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7634.214631] alg: No test for adler32 (adler32-zlib) [ 7635.350466] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 7635.659723] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 7637.351218] Key type lgssc registered [ 7638.554348] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7646.405144] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7646.421184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7647.889955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7652.955459] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7658.470446] LustreError: 260124:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7658.496036] LustreError: 260124:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7661.521793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7661.884688] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7666.732754] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7674.534727] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7680.098866] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7686.525929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7686.887275] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7692.361037] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7693.039980] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7698.801045] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7699.746888] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7703.528946] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7703.529852] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7703.558166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7706.659412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7707.692971] Lustre: server umount lustre-OST0000 complete [ 7711.716900] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7711.740648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7711.746066] Lustre: Skipped 2 previous similar messages [ 7716.838512] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7716.846052] Lustre: Skipped 1 previous similar message [ 7716.934858] Lustre: server umount lustre-MDT0000 complete [ 7720.175728] LustreError: 260105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279151 with bad export cookie 2767609932247547198 [ 7720.179385] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7720.184256] LustreError: 260105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7720.465815] Lustre: server umount lustre-MDT0001 complete [ 7730.166728] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 7733.025687] Key type lgssc unregistered [ 7733.308968] LNet: 263709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7733.321894] LNetError: 263709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7733.338740] LNet: Removed LNI 192.168.204.153@tcp [ 7734.195807] Key type .llcrypt unregistered [ 7734.197851] Key type ._llcrypt unregistered [ 7749.673425] Key type ._llcrypt registered [ 7749.675192] Key type .llcrypt registered [ 7749.754327] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 7750.881447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7750.931315] alg: No test for adler32 (adler32-zlib) [ 7752.001711] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 7752.257338] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 7753.999196] Key type lgssc registered [ 7755.060523] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7762.440293] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7762.454368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7777.951595] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7783.071820] LustreError: 264903:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.153@tcp: failed processing log, type 4: rc = -110 [ 7809.761459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7815.682244] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 7822.903472] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7825.033326] Lustre: server umount lustre-OST0000 complete [ 7840.874754] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 7843.488661] Key type lgssc unregistered [ 7843.776589] LNet: 266537:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7843.784700] LNetError: 266537:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7843.809379] LNet: Removed LNI 192.168.204.153@tcp [ 7844.642132] Key type .llcrypt unregistered [ 7844.646338] Key type ._llcrypt unregistered [ 7887.650382] Key type ._llcrypt registered [ 7887.658670] Key type .llcrypt registered [ 7887.822724] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 7889.185864] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7889.203318] alg: No test for adler32 (adler32-zlib) [ 7890.252695] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 7890.560109] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 7892.208851] Key type lgssc registered [ 7893.690653] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7907.452394] loop0: detected capacity change from 0 to 204800000 [ 7907.616869] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7907.625511] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7907.990286] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7909.027539] loop0: detected capacity change from 0 to 204800000 [ 7909.280510] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7909.297357] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7909.816535] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7911.121398] loop0: detected capacity change from 0 to 204800000 [ 7911.304537] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7911.319657] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7911.421261] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7912.749290] loop0: detected capacity change from 0 to 204800000 [ 7912.975105] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7913.000734] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7913.149723] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7914.393591] loop0: detected capacity change from 0 to 204800000 [ 7914.415735] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7918.589326] loop0: detected capacity change from 0 to 204800000 [ 7918.640743] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7922.725962] loop0: detected capacity change from 0 to 204800000 [ 7922.753774] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7927.107885] loop0: detected capacity change from 0 to 204800000 [ 7927.144471] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7931.323826] loop0: detected capacity change from 0 to 204800000 [ 7931.455382] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7931.502429] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7931.579514] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7931.589756] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7931.601567] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7931.612538] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7931.626520] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7931.635555] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7931.648489] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7931.657112] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7931.665613] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7931.673492] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7931.681264] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7931.688681] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7931.696383] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7931.704654] Lustre: 269949:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7938.556548] Lustre: server umount MGS complete [ 7939.505319] loop0: detected capacity change from 0 to 204800000 [ 7939.724915] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7940.438395] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7940.502901] Lustre: lustre-MDT0000: Aborting client recovery [ 7941.685621] loop1: detected capacity change from 0 to 204800000 [ 7941.800476] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7941.849359] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7941.858825] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7941.866504] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7941.872456] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7941.878417] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7941.884242] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7941.889917] Lustre: 270381:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7941.901428] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7942.013321] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7942.043943] Lustre: lustre-MDT0001: Aborting client recovery [ 7943.098984] loop2: detected capacity change from 0 to 204800000 [ 7943.294297] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7943.491542] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7943.512209] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7943.537357] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7943.555976] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7943.598416] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7943.615693] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7943.648458] Lustre: 270460:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7943.799761] Lustre: Found index 0 for lustre-OST0000, updating log [ 7943.830782] Lustre: 270479:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7943.901370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7943.915351] Lustre: lustre-OST0000: Aborting recovery [ 7944.973258] loop3: detected capacity change from 0 to 204800000 [ 7945.127131] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7945.241965] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7945.263252] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7945.279040] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7945.303767] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7945.320161] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7945.340120] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7945.358754] Lustre: 270544:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7945.417884] Lustre: Found index 1 for lustre-OST0001, updating log [ 7945.471352] Lustre: 270550:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7945.578749] Lustre: lustre-OST0001: Aborting recovery [ 7949.690996] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7949.705020] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7949.735830] LustreError: 270325:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 7949.759126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7949.798140] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7951.715432] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7951.721097] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7953.983131] LustreError: 270292:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 697us: rc = -2 [ 7954.495666] LustreError: 270292:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 663us: rc = -2 [ 7954.513686] LustreError: 270292:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 7967.201718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7967.214312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7967.214945] Lustre: Skipped 1 previous similar message [ 7972.332287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7972.335244] Lustre: Skipped 5 previous similar messages [ 7973.265730] Lustre: server umount lustre-MDT0000 complete [ 7974.635622] LustreError: 270276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279406 with bad export cookie 3252931043000968409 [ 7974.682434] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7977.443495] LustreError: 270888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7977.446692] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7977.448266] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7977.448275] Lustre: Skipped 1 previous similar message [ 7977.473432] LustreError: 270888:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7977.539532] Lustre: Skipped 1 previous similar message [ 7979.498882] LustreError: 270289:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7979.499468] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7979.528796] LustreError: 270289:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7979.546674] Lustre: Skipped 2 previous similar messages [ 7981.256530] Lustre: server umount lustre-MDT0001 complete [ 7996.383902] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7996.526797] Lustre: server umount lustre-OST0000 complete [ 8003.961809] Lustre: server umount lustre-OST0001 complete [ 8013.226696] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:17:23 (1788279443) [ 8047.454521] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 8062.683679] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 8103.689156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8103.965508] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8103.994694] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8104.055541] Lustre: lustre-MDT0000: new disk, initializing [ 8104.121606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8104.128935] Lustre: Skipped 1 previous similar message [ 8104.141691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8109.265420] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8121.531419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8121.757768] Lustre: 274637:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8121.885619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8121.891164] Lustre: Skipped 1 previous similar message [ 8121.970169] Lustre: lustre-MDT0001: new disk, initializing [ 8122.125872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8122.197205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8122.215648] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8127.219784] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8139.121310] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8146.635215] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8153.952271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8154.233607] Lustre: lustre-OST0000: new disk, initializing [ 8154.238395] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8154.246141] Lustre: 276608:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8154.345828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8161.212520] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8164.413102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8164.421479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8164.506102] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8171.056396] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8177.613594] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8190.346385] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8196.654825] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8201.761824] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8246.753170] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8246.759767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8246.770370] LustreError: Skipped 1 previous similar message [ 8246.774393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8246.791905] Lustre: Skipped 1 previous similar message [ 8246.817477] Lustre: Skipped 1 previous similar message [ 8251.879684] Lustre: server umount lustre-OST0000 complete [ 8256.994786] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8257.028608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8257.045798] Lustre: Skipped 1 previous similar message [ 8263.176571] Lustre: server umount lustre-MDT0000 complete [ 8267.233462] LustreError: 276556:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8267.262972] LustreError: 276556:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8267.780620] LustreError: 274630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279699 with bad export cookie 3252931043000971111 [ 8267.789970] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8267.794886] LustreError: 274630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8268.258935] Lustre: server umount lustre-MDT0001 complete [ 8278.126190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8283.135256] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8290.028748] Lustre: server umount MGS complete [ 8304.052861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8304.521638] LustreError: 280477:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8304.544744] LustreError: 280490:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8304.655503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8309.728684] LustreError: 280478:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8310.190090] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8319.971617] LustreError: 280477:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8319.996286] LustreError: 280477:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8320.334587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8325.138964] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8333.656231] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8340.877873] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8350.324044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8350.976172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8350.992287] Lustre: Skipped 1 previous similar message [ 8359.881870] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8370.298082] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8408.544422] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8408.549706] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8408.566919] Lustre: Skipped 1 previous similar message [ 8408.574078] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8408.578262] Lustre: Skipped 2 previous similar messages [ 8411.645716] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8414.502063] Lustre: server umount lustre-OST0000 complete [ 8421.863329] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8421.880487] Lustre: Skipped 2 previous similar messages [ 8424.788485] Lustre: server umount lustre-MDT0000 complete [ 8426.988614] LustreError: 280478:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8427.024881] LustreError: 280478:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8429.085813] LustreError: 281214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788279860 with bad export cookie 3252931043000972938 [ 8429.099160] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8429.105478] LustreError: 281214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8429.418788] Lustre: server umount lustre-MDT0001 complete [ 8445.267911] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 8449.766258] Key type lgssc unregistered [ 8450.288949] LNet: 284429:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8450.309412] LNetError: 284429:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8450.363739] LNet: Removed LNI 192.168.204.153@tcp [ 8451.745284] Key type .llcrypt unregistered [ 8451.746994] Key type ._llcrypt unregistered [ 8494.527429] Key type ._llcrypt registered [ 8494.529634] Key type .llcrypt registered [ 8494.625967] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 8511.651643] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 8512.971878] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8512.998120] alg: No test for adler32 (adler32-zlib) [ 8514.273691] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 8514.490591] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 8516.193089] Key type lgssc registered [ 8517.713492] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8559.346785] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8559.383270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8560.571815] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8560.630164] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8560.765519] Lustre: lustre-MDT0000: new disk, initializing [ 8560.926504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8560.946538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8566.745443] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8579.241472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8579.421565] Lustre: 288392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8579.478130] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8579.482732] Lustre: Skipped 1 previous similar message [ 8579.599533] Lustre: lustre-MDT0001: new disk, initializing [ 8579.708846] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8579.752076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8579.765583] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8586.299721] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8597.757665] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8605.707374] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8614.658844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8615.062908] Lustre: lustre-OST0000: new disk, initializing [ 8615.066927] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8615.074180] Lustre: 290359:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8615.190056] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8622.032256] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8622.695609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8622.709505] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8622.881143] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8632.786823] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8639.898194] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8651.382969] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8660.224370] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8668.424201] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8720.353428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8720.362818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8720.393695] Lustre: Skipped 1 previous similar message [ 8720.430685] Lustre: Skipped 1 previous similar message [ 8725.478198] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8725.508738] Lustre: Skipped 1 previous similar message [ 8730.604175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8730.618490] Lustre: Skipped 1 previous similar message [ 8732.128488] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8732.455622] Lustre: server umount lustre-OST0000 complete [ 8738.277345] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8738.286267] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8738.307035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8740.835951] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8744.591919] Lustre: server umount lustre-MDT0000 complete [ 8750.089216] LustreError: 288383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788280181 with bad export cookie 14988384700376210669 [ 8750.097061] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8750.110241] LustreError: 288383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8750.912231] Lustre: server umount lustre-MDT0001 complete [ 8763.067760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8768.954359] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8774.971778] Lustre: server umount MGS complete [ 8789.085285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8789.705387] LustreError: 294285:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8789.737613] LustreError: 294285:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8789.773159] LustreError: 294299:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8789.897387] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8795.106891] LustreError: 294286:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8795.841314] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8800.230376] LustreError: 294285:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8805.354300] LustreError: 294286:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8806.547863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8807.013097] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8813.558369] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8823.499698] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8833.481348] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8842.778119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8843.486901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8852.012756] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 8860.873595] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8890.848680] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8890.850494] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8890.862967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8890.862975] Lustre: Skipped 2 previous similar messages [ 8890.885845] Lustre: Skipped 1 previous similar message [ 8896.603749] Lustre: server umount lustre-OST0000 complete [ 8904.170240] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8904.181760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8904.198376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8904.203751] Lustre: Skipped 3 previous similar messages [ 8907.525692] Lustre: server umount lustre-MDT0000 complete [ 8911.267860] LustreError: 294267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788280342 with bad export cookie 14988384700376212615 [ 8911.273631] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8911.276764] LustreError: 294267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8911.739939] Lustre: server umount lustre-MDT0001 complete [ 8925.668044] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [ 8929.483410] Key type lgssc unregistered [ 8929.951805] LNet: 298237:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8929.957417] LNetError: 298237:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8929.980444] LNet: Removed LNI 192.168.204.153@tcp [ 8930.977811] Key type .llcrypt unregistered [ 8930.980784] Key type ._llcrypt unregistered [ 8956.358202] Key type ._llcrypt registered [ 8956.360567] Key type .llcrypt registered [ 8956.501161] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 8972.055558] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 8973.400812] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8973.654715] alg: No test for adler32 (adler32-zlib) [ 8974.663245] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [ 8974.961546] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [ 8976.688299] Key type lgssc registered [ 8977.850647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9000.564761] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 9014.748404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9014.767698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9016.013818] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9016.047933] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9016.150670] Lustre: lustre-MDT0000: new disk, initializing [ 9016.289689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9016.347475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9020.590418] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 9030.291568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9030.592743] Lustre: lustre-OST0000: new disk, initializing [ 9030.596260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9030.600358] Lustre: Skipped 1 previous similar message [ 9030.606783] Lustre: 302288:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9030.702700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9032.209756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9032.232242] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9032.310187] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9038.231896] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [ 9902.565715] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9902.594849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9903.152617] Lustre: server umount lustre-MDT0000 complete [ 9908.300911] LustreError: 301477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788281339 with bad export cookie 6317671144122811510 [ 9908.324778] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9908.331043] LustreError: 301477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9908.530820] Lustre: server umount lustre-OST0000 complete [ 9938.837507] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [ 9951.544584] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [ 9994.510710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9994.898586] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9994.936977] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9995.095873] Lustre: lustre-MDT0000: new disk, initializing [ 9995.262821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9995.290730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10001.735860] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10015.223633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10015.407591] Lustre: 306633:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10015.491218] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10015.496772] Lustre: Skipped 1 previous similar message [10015.574147] Lustre: lustre-MDT0001: new disk, initializing [10015.656752] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10015.700507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10015.715634] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10020.582549] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10030.765247] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10036.398862] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10044.760401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10045.295435] Lustre: lustre-OST0000: new disk, initializing [10045.310760] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10045.328881] Lustre: 308602:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10045.454979] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10051.153343] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10051.172601] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10051.507030] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10054.119738] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10067.777696] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10074.637793] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10074.995661] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10081.218510] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10081.552908] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10087.393202] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10087.422621] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10087.438256] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10090.275093] Lustre: server umount lustre-OST0000 complete [10097.632647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10097.678749] Lustre: Skipped 1 previous similar message [10097.705745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10097.717363] Lustre: Skipped 2 previous similar messages [10102.758515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10102.773439] Lustre: Skipped 1 previous similar message [10107.876355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10107.881938] Lustre: Skipped 1 previous similar message [10109.410083] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10109.743053] Lustre: server umount lustre-MDT0000 complete [10113.944677] LustreError: 308601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788281545 with bad export cookie 6317671144125752336 [10113.948914] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10113.958498] LustreError: 308601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10114.433492] Lustre: server umount lustre-MDT0001 complete [10118.208837] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10120.017778] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:52:30 (1788281550) [10126.885571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10127.386806] LustreError: 310918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10127.407712] LustreError: 310918:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10127.512439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10132.975734] LustreError: 310919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10133.340871] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10137.065342] LustreError: 310918:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10142.181044] LustreError: 310919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10142.917609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10143.263929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10148.403844] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10157.879450] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10164.227777] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10172.473233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10173.030895] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10180.597719] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10189.122968] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10197.046829] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10197.291702] Lustre: lustre-OST0001: new disk, initializing [10197.296504] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10197.312894] Lustre: 313757:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10197.454852] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10205.253441] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10206.355910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10206.364218] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10206.476825] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10206.502996] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10207.536046] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10207.551697] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10208.546867] LustreError: 314155:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10210.726166] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10210.748890] LustreError: 314146:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10214.951417] LustreError: 314155:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10214.969191] LustreError: 314155:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10217.864990] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10218.097311] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10223.209200] LustreError: 314155:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10223.225608] LustreError: 314155:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10224.459153] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0cc47960000.ost_server_uuid 50 [10226.356644] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0cc47960000.ost_server_uuid in FULL state after 0 sec [10232.316756] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0cc47960000.ost_server_uuid 50 [10233.977959] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0cc47960000.ost_server_uuid in FULL state after 0 sec [10235.665852] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10251.236112] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10251.239915] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10251.247965] LustreError: Skipped 2 previous similar messages [10251.263271] Lustre: Skipped 1 previous similar message [10251.265801] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10251.281835] Lustre: Skipped 1 previous similar message [10255.498449] Lustre: server umount lustre-OST0001 complete [10257.903580] LustreError: 312742:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10257.934794] LustreError: 312742:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10263.011156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10263.031419] Lustre: Skipped 1 previous similar message [10268.132171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10268.133270] LustreError: 312740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10268.144502] Lustre: Skipped 6 previous similar messages [10268.162665] LustreError: 312740:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10268.942828] Lustre: server umount lustre-OST0000 complete [10273.251754] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10273.263327] Lustre: Skipped 2 previous similar messages [10279.494923] Lustre: server umount lustre-MDT0000 complete [10283.836648] LustreError: 310900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788281715 with bad export cookie 6317671144125753218 [10283.841136] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10283.854974] LustreError: 310900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10284.427310] Lustre: server umount lustre-MDT0001 complete [10297.188319] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [10301.026303] Key type lgssc unregistered [10301.323645] LNet: 316450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10301.333382] LNetError: 316450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10301.359955] LNet: Removed LNI 192.168.204.153@tcp [10302.366445] Key type .llcrypt unregistered [10302.370845] Key type ._llcrypt unregistered [10329.972609] Key type ._llcrypt registered [10329.975262] Key type .llcrypt registered [10330.084193] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [10331.585165] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10331.632911] alg: No test for adler32 (adler32-zlib) [10332.780632] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [10332.972682] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [10334.664848] Key type lgssc registered [10335.949277] Lustre: Echo OBD driver; http://www.lustre.org/ [10346.268565] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10346.314641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10347.876671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10353.207214] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10358.245786] LustreError: 317874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10358.264212] LustreError: 317874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10362.448076] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10363.365990] LustreError: 317873:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10368.503046] LustreError: 317874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10370.798104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10371.360255] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10376.748374] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10387.403708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10387.841424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10392.943704] LustreError: 319555:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10392.968805] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10395.149399] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10401.250699] LustreError: 319556:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10401.270996] LustreError: 319556:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10403.211836] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10411.070516] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10411.393107] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10416.644655] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10417.807479] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10425.897543] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10433.788423] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10433.986541] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10539.491544] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10539.497791] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10539.508732] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10539.513278] Lustre: Skipped 1 previous similar message [10543.855158] Lustre: server umount lustre-OST0001 complete [10544.632133] LustreError: 319836:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10544.664464] LustreError: 319836:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10554.848946] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10554.868730] Lustre: Skipped 1 previous similar message [10554.883601] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10554.888730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10554.892784] LustreError: Skipped 1 previous similar message [10554.907732] Lustre: Skipped 1 previous similar message [10558.723815] Lustre: server umount lustre-OST0000 complete [10565.091797] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10565.104254] Lustre: Skipped 1 previous similar message [10565.108897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10565.112925] Lustre: Skipped 1 previous similar message [10569.404443] Lustre: server umount lustre-MDT0000 complete [10570.210105] LustreError: 317873:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10570.219840] LustreError: 317873:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10573.386282] LustreError: 317853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788282004 with bad export cookie 3087857543880630746 [10573.394483] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10573.401348] LustreError: 317853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10573.740709] Lustre: server umount lustre-MDT0001 complete [10584.212996] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [10586.992298] Key type lgssc unregistered [10587.301473] LNet: 322821:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10587.309975] LNetError: 322821:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10587.342481] LNet: Removed LNI 192.168.204.153@tcp [10588.160105] Key type .llcrypt unregistered [10588.165920] Key type ._llcrypt unregistered [10631.334135] Key type ._llcrypt registered [10631.338404] Key type .llcrypt registered [10631.483767] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [10632.490967] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10632.522047] alg: No test for adler32 (adler32-zlib) [10633.679734] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [10633.961690] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [10635.695209] Key type lgssc registered [10636.805908] Lustre: Echo OBD driver; http://www.lustre.org/ [10646.526259] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10646.539941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10648.124978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10652.813473] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10658.284988] LustreError: 325124:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10658.307663] LustreError: 325124:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10662.347452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10662.628724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10667.751386] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10670.613226] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10676.641851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10676.938139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10682.721931] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10688.235249] LustreError: 326612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10691.569652] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10691.802790] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10692.862743] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10692.864514] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10698.060969] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10707.416411] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10712.975527] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10724.320521] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10724.330889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10724.346792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10728.936350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10728.941376] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10728.960040] Lustre: Skipped 2 previous similar messages [10730.600191] Lustre: server umount lustre-MDT0000 complete [10734.052227] LustreError: 327895:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10734.083798] LustreError: 327895:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10734.610058] LustreError: 325103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788282166 with bad export cookie 2409233769747231012 [10734.611957] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10734.622420] LustreError: 325103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10734.974375] Lustre: server umount lustre-MDT0001 complete [10749.527644] Lustre: server umount lustre-OST0000 complete [10762.796537] Lustre: server umount lustre-OST0001 complete [10781.135225] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [10792.004214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10792.353578] LustreError: 330391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10792.444937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10797.519595] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10797.543630] LustreError: 330392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10806.781060] LustreError: 330391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10806.831901] LustreError: 330391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10807.739923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10808.321226] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10813.606273] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10817.241223] Lustre: 331531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10825.528816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10825.932293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10832.419967] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10834.157017] LustreError: 331888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10837.241238] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10840.920656] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10846.716682] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10847.794200] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [10855.908527] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10859.586266] Lustre: 333403:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10864.611706] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10864.620367] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10864.659421] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10864.669131] Lustre: Skipped 3 previous similar messages [10867.184820] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10867.185257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10867.217448] Lustre: Skipped 2 previous similar messages [10867.240036] Lustre: Skipped 3 previous similar messages [10870.586462] Lustre: server umount lustre-MDT0000 complete [10872.293943] LustreError: 330387:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10872.324615] LustreError: 330387:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10875.014357] LustreError: 330371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788282306 with bad export cookie 2409233769747232713 [10875.022160] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10875.035632] LustreError: 330371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10875.558696] Lustre: server umount lustre-MDT0001 complete [10890.324830] Lustre: server umount lustre-OST0000 complete [10893.151770] Lustre: 324386:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788282308/real 1788282308] req@ffff997983cb0e00 x1875149654702976/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788282324 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10893.178665] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10894.801640] Lustre: server umount lustre-OST0001 complete [10898.833779] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10900.573198] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10902.241603] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:05:32 (1788282332) [10929.955592] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [10940.655943] loop0: detected capacity change from 0 to 1099511627776 [10941.390764] SGI XFS with ACLs, security attributes, quota, no debug enabled [10941.400209] *** VALIDATE xfs *** [10941.414847] XFS (loop0): Mounting V5 Filesystem [10941.439963] XFS (loop0): Ending clean mount [10942.984345] loop1: detected capacity change from 0 to 18691697672192 [11072.625833] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [11082.772393] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [11121.045960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11121.284541] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11121.327806] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11121.436875] Lustre: lustre-MDT0000: new disk, initializing [11121.542940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11121.548852] Lustre: Skipped 1 previous similar message [11121.561898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11126.363402] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11139.182489] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11139.363687] Lustre: 338547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11139.435472] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11139.445758] Lustre: Skipped 1 previous similar message [11139.547229] Lustre: lustre-MDT0001: new disk, initializing [11139.715264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11139.733336] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11144.595430] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11158.844643] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11165.573515] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11174.677467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11174.985560] Lustre: lustre-OST0000: new disk, initializing [11174.988723] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11174.993834] Lustre: 340515:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11176.281890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11176.307331] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11176.473067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11183.210491] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11193.482558] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11198.123755] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11198.276357] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11202.482592] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11202.686322] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11207.137141] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11207.141609] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11207.150589] LustreError: Skipped 1 previous similar message [11207.152627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11207.170253] Lustre: Skipped 1 previous similar message [11211.104040] Lustre: server umount lustre-OST0000 complete [11216.352925] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11216.362253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11216.367658] Lustre: Skipped 1 previous similar message [11221.073358] Lustre: server umount lustre-MDT0000 complete [11222.497115] LustreError: 338558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11222.513122] LustreError: 338558:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11225.228112] LustreError: 338540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788282656 with bad export cookie 2409233769747234260 [11225.232460] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11225.236441] LustreError: 338540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11225.630224] Lustre: server umount lustre-MDT0001 complete [11233.559924] XFS (loop0): Unmounting Filesystem [11238.164756] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:11:08 (1788282668) [11246.225895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11246.924081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11246.935088] Lustre: Skipped 2 previous similar messages [11251.105910] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11259.598325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11264.401381] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11272.087517] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11278.278900] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11285.894227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11292.442747] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11299.552719] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11313.129018] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11313.143072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11313.156941] Lustre: Skipped 2 previous similar messages [11313.164083] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11313.169479] Lustre: Skipped 2 previous similar messages [11318.051390] Lustre: server umount lustre-OST0000 complete [11321.824109] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11321.833117] LustreError: Skipped 1 previous similar message [11328.054395] Lustre: server umount lustre-MDT0000 complete [11331.855972] LustreError: 343058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788282763 with bad export cookie 2409233769747235142 [11331.862580] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11331.874214] LustreError: 343058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11332.403055] Lustre: server umount lustre-MDT0001 complete [11343.258852] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [11346.572507] Key type lgssc unregistered [11346.883615] LNet: 346710:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11346.898976] LNetError: 346710:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11347.942442] LNet: Removed LNI 192.168.204.153@tcp [11349.094184] Key type .llcrypt unregistered [11349.099311] Key type ._llcrypt unregistered [11379.527594] Key type ._llcrypt registered [11379.529886] Key type .llcrypt registered [11379.680807] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [11381.192071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11381.260241] alg: No test for adler32 (adler32-zlib) [11382.456512] Lustre: Lustre: Build Version: 2.17.57_64_g65721f0 [11382.708444] LNet: Added LNI 192.168.204.153@tcp [8/256/0/180] [11384.495335] Key type lgssc registered [11386.248312] Lustre: Echo OBD driver; http://www.lustre.org/ [11396.613600] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11396.657470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11398.448081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11402.661332] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11407.854314] LustreError: 348278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11407.876656] LustreError: 348278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11412.967869] LustreError: 348279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11414.223593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11415.286278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11421.279423] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11430.669972] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11436.881582] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11445.079045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11445.506204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11451.652820] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11458.833927] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11476.960248] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11476.968648] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11476.986794] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11477.475388] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11477.492478] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11477.498870] Lustre: Skipped 1 previous similar message [11479.855772] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.53@tcp (stopping) [11481.999168] Lustre: server umount lustre-MDT0001 complete [11482.593091] LustreError: 348279:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11482.609415] LustreError: 348279:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11484.937740] LustreError: 350768:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.53@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11490.045323] LustreError: 350768:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.53@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11490.070214] LustreError: 350768:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11492.740448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11492.877784] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11492.882335] Lustre: Found index 1 for lustre-MDT0001, updating log [11492.959600] Lustre: 348266:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11493.276211] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11498.484055] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11498.933697] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11500.530589] LustreError: 351823:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11500.538550] LustreError: 351823:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11500.549744] LustreError: 348271:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11500.639460] LustreError: 351832:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.153@tcp: cfg command failed: rc = -17 [11500.651760] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11500.651760] [11510.570417] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.53@tcp (stopping) [11513.826093] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11515.662932] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.53@tcp (stopping) [11515.694127] Lustre: Skipped 2 previous similar messages [11516.218429] Lustre: server umount lustre-MDT0001 complete [11518.954289] LustreError: 348278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11518.989388] LustreError: 348278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11529.113130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11529.252166] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11529.258414] Lustre: Found index 1 for lustre-MDT0001, updating log [11529.312590] Lustre: 348266:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11529.650257] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11534.585941] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11534.842877] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11538.937221] LustreError: 352980:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11538.950994] LustreError: 352980:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11538.959673] LustreError: 348271:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11538.972230] LustreError: 348271:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11539.052568] LustreError: 352983:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11539.066465] LustreError: 352983:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11539.075660] LustreError: 352983:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.153@tcp: cfg command failed: rc = -17 [11539.091448] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.153@tcp [11539.091448] [11842.035475] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11842.068036] Lustre: Skipped 1 previous similar message [11842.106239] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11842.113814] Lustre: Skipped 1 previous similar message [11847.835157] Lustre: server umount lustre-MDT0001 complete [11848.461335] LustreError: 348273:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.53@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11848.489257] LustreError: 348273:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [11859.099843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11859.258884] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11859.264398] Lustre: Found index 1 for lustre-MDT0001, updating log [11859.329483] Lustre: 348267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11859.677696] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11864.592292] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [11865.083696] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11868.156824] LustreError: 361053:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11868.163964] LustreError: 361053:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11868.171487] LustreError: 348271:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11868.179410] LustreError: 348271:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11868.257394] LustreError: 361056:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11868.283396] LustreError: 361056:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.153@tcp: cfg command failed: rc = -114 [11868.300850] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11868.300850] [12187.631281] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12187.636433] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12187.645846] Lustre: Skipped 1 previous similar message [12187.648447] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12187.708319] Lustre: Skipped 5 previous similar messages [12193.248042] Lustre: server umount lustre-MDT0000 complete [12197.863082] LustreError: 348274:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12197.896441] LustreError: 348274:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [12198.910605] LustreError: 351487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788283630 with bad export cookie 779272159743857429 [12198.918702] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12198.918837] LustreError: 351487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12202.987939] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12203.001333] Lustre: Skipped 2 previous similar messages [12205.477139] Lustre: server umount lustre-MDT0001 complete [12210.484399] Lustre: server umount lustre-OST0000 complete [12221.249437] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [12231.058305] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [12276.199411] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 13:28:26 (1788283706) [12283.794323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12283.981714] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12284.005481] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12284.065100] Lustre: lustre-MDT0000: new disk, initializing [12284.155758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12284.187528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12288.912982] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [12299.500373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12299.754542] Lustre: 371435:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12299.853789] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12299.857952] Lustre: Skipped 1 previous similar message [12299.956233] Lustre: lustre-MDT0001: new disk, initializing [12300.123749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12300.175500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12300.190776] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12304.659660] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [12315.928841] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12322.661012] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12329.218271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12329.417373] Lustre: lustre-OST0000: new disk, initializing [12329.420894] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12329.436295] Lustre: 373403:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12329.532400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12337.258077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12337.271947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12337.445209] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_default_debug -1 all [12337.478230] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12347.799362] Lustre: DEBUG MARKER: oleg453-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12356.583750] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12356.594158] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12356.611335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12356.616462] Lustre: Skipped 5 previous similar messages [12358.337636] Lustre: server umount lustre-MDT0000 complete [12361.743061] LustreError: 371426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788283793 with bad export cookie 779272159743859382 [12361.745934] LustreError: MGC192.168.204.153@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12361.753555] LustreError: 372239:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12361.760159] LustreError: 371426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12362.191860] Lustre: server umount lustre-MDT0001 complete [12377.257518] Lustre: server umount lustre-OST0000 complete [12409.153806] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing set_hostid [12420.511266] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing load_modules_local [12456.535473] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12174 sec ========= 13:31:26 (1788283886) [12458.420833] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:31:28 (1788283888) === [12462.885305] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:31:32 (1788283892) === [12514.303590] Lustre: DEBUG MARKER: oleg453-server.virtnet: executing unload_modules_local [12518.277161] Key type lgssc unregistered [12518.589964] LNet: 379777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12518.613585] LNetError: 379777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12518.643429] LNet: Removed LNI 192.168.204.153@tcp [12519.552886] Key type .llcrypt unregistered [12519.555172] Key type ._llcrypt unregistered