[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 615171831 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002269] x2apic enabled [ 0.003011] Switched APIC routing to physical x2apic. [ 0.004014] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007022] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008032] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009019] pid_max: default: 32768 minimum: 301 [ 0.010163] LSM: Security Framework initializing [ 0.011081] Yama: becoming mindful. [ 0.012052] SELinux: Initializing. [ 0.014032] *** VALIDATE selinux *** [ 0.023770] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029136] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030243] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031155] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032159] *** VALIDATE tmpfs *** [ 0.034123] *** VALIDATE proc *** [ 0.035357] *** VALIDATE cgroup *** [ 0.037008] *** VALIDATE cgroup2 *** [ 0.038364] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039194] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040020] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042034] Spectre V2 : User space: Vulnerable [ 0.043014] Speculative Store Bypass: Vulnerable [ 0.046436] debug: unmapping init [mem 0xffffffffafc59000-0xffffffffafc60fff] [ 0.049265] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050887] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051032] ... version: 2 [ 0.052019] ... bit width: 48 [ 0.053022] ... generic registers: 4 [ 0.054025] ... value mask: 0000ffffffffffff [ 0.055023] ... max period: 00007fffffffffff [ 0.056020] ... fixed-purpose events: 3 [ 0.057018] ... event mask: 000000070000000f [ 0.058406] rcu: Hierarchical SRCU implementation. [ 0.060868] smp: Bringing up secondary CPUs ... [ 0.061599] x86: Booting SMP configuration: [ 0.062061] .... node #0, CPUs: #1 #2 #3 [ 0.065437] smp: Brought up 1 node, 4 CPUs [ 0.067021] smpboot: Max logical packages: 1 [ 0.068029] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.149514] node 0 deferred pages initialised in 77ms [ 0.152145] devtmpfs: initialized [ 0.153290] x86/mm: Memory block size: 128MB [ 0.156088] gcov: version magic: 0x41383552 [ 0.158356] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.159094] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.160324] pinctrl core: initialized pinctrl subsystem [ 0.161244] [ 0.161788] ************************************************************* [ 0.162026] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163013] ** ** [ 0.164025] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.165018] ** ** [ 0.166014] ** This means that this kernel is built to expose internal ** [ 0.167017] ** IOMMU data structures, which may compromise security on ** [ 0.168012] ** your system. ** [ 0.169022] ** ** [ 0.170017] ** If you see this message and you are not debugging the ** [ 0.171014] ** kernel, report this immediately to your vendor! ** [ 0.172013] ** ** [ 0.173016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.174014] ************************************************************* [ 0.175567] NET: Registered protocol family 16 [ 0.176321] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.177074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.178065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.179460] cpuidle: using governor menu [ 0.180000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.182499] PCI: Using configuration type 1 for base access [ 0.184095] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.192157] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.193040] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.195069] cryptd: max_cpu_qlen set to 1000 [ 0.197024] ACPI: Added _OSI(Module Device) [ 0.199012] ACPI: Added _OSI(Processor Device) [ 0.201050] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.202015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.207297] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.212373] ACPI: Interpreter enabled [ 0.213085] ACPI: PM: (supports S0 S3 S4 S5) [ 0.215017] ACPI: Using IOAPIC for interrupt routing [ 0.217156] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.220404] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.230928] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.233062] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.235019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.238074] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.242000] acpiphp: Slot [2] registered [ 0.242000] acpiphp: Slot [5] registered [ 0.243186] acpiphp: Slot [6] registered [ 0.244100] acpiphp: Slot [7] registered [ 0.245076] acpiphp: Slot [8] registered [ 0.246000] acpiphp: Slot [9] registered [ 0.246000] acpiphp: Slot [10] registered [ 0.248141] acpiphp: Slot [3] registered [ 0.249115] acpiphp: Slot [4] registered [ 0.250059] acpiphp: Slot [11] registered [ 0.251000] acpiphp: Slot [12] registered [ 0.251000] acpiphp: Slot [13] registered [ 0.252128] acpiphp: Slot [14] registered [ 0.254182] acpiphp: Slot [15] registered [ 0.255000] acpiphp: Slot [16] registered [ 0.255000] acpiphp: Slot [17] registered [ 0.255000] acpiphp: Slot [18] registered [ 0.255000] acpiphp: Slot [19] registered [ 0.255000] acpiphp: Slot [20] registered [ 0.255000] acpiphp: Slot [21] registered [ 0.256080] acpiphp: Slot [22] registered [ 0.256978] acpiphp: Slot [23] registered [ 0.257000] acpiphp: Slot [24] registered [ 0.257078] acpiphp: Slot [25] registered [ 0.258000] acpiphp: Slot [26] registered [ 0.259075] acpiphp: Slot [27] registered [ 0.259989] acpiphp: Slot [28] registered [ 0.261076] acpiphp: Slot [29] registered [ 0.261898] acpiphp: Slot [30] registered [ 0.262076] acpiphp: Slot [31] registered [ 0.262950] PCI host bridge to bus 0000:00 [ 0.264022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.265031] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.268035] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.270075] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.271028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.273057] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.274247] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.276884] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.279085] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.288786] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.293548] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.296036] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.298024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.301032] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.304497] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.306896] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.309055] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.311814] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.315030] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.325857] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.329029] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.333261] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.340095] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.348025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.366022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.377360] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.385032] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.390022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.413025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.427334] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.435028] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.445035] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.462035] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.475336] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.486099] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.495026] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.524034] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.546031] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.555025] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.561034] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.577030] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.600042] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.624034] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.635035] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.679041] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.697000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.699373] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.701330] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.702326] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.704247] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.708257] iommu: Default domain type: Passthrough [ 0.711762] SCSI subsystem initialized [ 0.714378] ACPI: bus type USB registered [ 0.716088] usbcore: registered new interface driver usbfs [ 0.717108] usbcore: registered new interface driver hub [ 0.718099] usbcore: registered new device driver usb [ 0.719107] pps_core: LinuxPPS API ver. 1 registered [ 0.720008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.723075] PTP clock support registered [ 0.724237] EDAC MC: Ver: 3.0.0 [ 0.726200] PCI: Using ACPI for IRQ routing [ 0.728283] NetLabel: Initializing [ 0.730018] NetLabel: domain hash size = 128 [ 0.732018] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.734144] NetLabel: unlabeled traffic allowed by default [ 0.737205] vgaarb: loaded [ 0.739459] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.741036] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.750062] clocksource: Switched to clocksource kvm-clock [ 0.863044] VFS: Disk quotas dquot_6.6.0 [ 0.864601] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.867419] *** VALIDATE ramfs *** [ 0.868748] *** VALIDATE hugetlbfs *** [ 0.870460] pnp: PnP ACPI init [ 0.872992] pnp: PnP ACPI: found 6 devices [ 0.890992] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.894528] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.896149] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.897621] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.899610] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.901302] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.903318] NET: Registered protocol family 2 [ 0.905427] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.909286] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.912133] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.916321] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.919285] TCP: Hash tables configured (established 65536 bind 65536) [ 0.921415] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.923653] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.926056] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.928334] NET: Registered protocol family 1 [ 0.931052] RPC: Registered named UNIX socket transport module. [ 0.933636] RPC: Registered udp transport module. [ 0.935413] RPC: Registered tcp transport module. [ 0.937179] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.939449] NET: Registered protocol family 44 [ 0.940972] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.942698] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.944739] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.946848] PCI: CLS 0 bytes, default 64 [ 0.948553] Unpacking initramfs... [ 2.449724] debug: unmapping init [mem 0xffff91d1bcc54000-0xffff91d1bffbffff] [ 2.453697] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.456188] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.459208] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.965577] Initialise system trusted keyrings [ 2.967062] Key type blacklist registered [ 2.968900] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.978523] zbud: loaded [ 2.982070] *** VALIDATE nfs *** [ 2.983397] *** VALIDATE nfs4 *** [ 2.985518] pstore: using deflate compression [ 2.995102] Platform Keyring initialized [ 3.108220] NET: Registered protocol family 38 [ 3.109500] Key type asymmetric registered [ 3.110555] Asymmetric key parser 'x509' registered [ 3.112019] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.114368] io scheduler mq-deadline registered [ 3.115551] io scheduler kyber registered [ 3.116805] io scheduler bfq registered [ 3.118118] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.120231] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.122259] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.124347] ACPI: Power Button [PWRF] [ 3.128725] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.134309] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.146599] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.154327] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.169152] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.197610] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.225377] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.229803] Non-volatile memory driver v1.3 [ 3.231298] Linux agpgart interface v0.103 [ 3.265946] virtio_blk virtio1: [vda] 150616 512-byte logical blocks (77.1 MB/73.5 MiB) [ 3.268970] vda: detected capacity change from 0 to 77115392 [ 3.286220] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.289399] vdb: detected capacity change from 0 to 1073741824 [ 3.309632] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.313584] vdc: detected capacity change from 0 to 2621440000 [ 3.337669] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.341326] vdd: detected capacity change from 0 to 2621440000 [ 3.367199] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.371356] vde: detected capacity change from 0 to 4294967296 [ 3.388769] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.392601] vdf: detected capacity change from 0 to 4294967296 [ 3.402121] libphy: Fixed MDIO Bus: probed [ 3.409352] usbcore: registered new interface driver usbserial_generic [ 3.413794] usbserial: USB Serial support registered for generic [ 3.416198] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.421114] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.422927] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.425770] mousedev: PS/2 mouse device common for all mice [ 3.429404] rtc_cmos 00:05: RTC can wake from S4 [ 3.432192] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.433784] rtc_cmos 00:05: registered as rtc0 [ 3.438567] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.441649] intel_pstate: CPU model not supported [ 3.444891] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.448581] hid: raw HID events driver (C) Jiri Kosina [ 3.451181] usbcore: registered new interface driver usbhid [ 3.453615] usbhid: USB HID core driver [ 3.455697] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.455715] drop_monitor: Initializing network drop monitor service [ 3.461939] Initializing XFRM netlink socket [ 3.464477] NET: Registered protocol family 10 [ 3.468668] Segment Routing with IPv6 [ 3.470232] NET: Registered protocol family 17 [ 3.472519] mpls_gso: MPLS GSO support [ 3.478666] RAS: Correctable Errors collector initialized. [ 3.480863] AVX version of gcm_enc/dec engaged. [ 3.482558] AES CTR mode by8 optimization enabled [ 3.569231] sched_clock: Marking stable (3569199939, 0)->(4661005974, -1091806035) [ 3.572660] registered taskstats version 1 [ 3.574647] Loading compiled-in X.509 certificates [ 3.576975] zswap: loaded using pool lzo/zbud [ 3.603700] Key type big_key registered [ 3.617123] Key type encrypted registered [ 3.618825] ima: No TPM chip found, activating TPM-bypass! [ 3.621173] ima: Allocated hash algorithm: sha1 [ 3.623048] ima: No architecture policies found [ 3.625132] evm: Initialising EVM extended attributes: [ 3.626954] evm: security.selinux [ 3.628347] evm: security.ima [ 3.629398] evm: security.capability [ 3.630964] evm: HMAC attrs: 0x1 [ 3.634070] rtc_cmos 00:05: setting system clock to 2026-10-02 12:57:16 UTC (1790945836) [ 3.640610] debug: unmapping init [mem 0xffffffffb0c03000-0xffffffffb0dfffff] [ 3.643904] debug: unmapping init [mem 0xffffffffaf982000-0xffffffffafc58fff] [ 3.652107] Write protecting the kernel read-only data: 28672k [ 3.655704] debug: unmapping init [mem 0xffffffffae003000-0xffffffffae1fffff] [ 3.658976] debug: unmapping init [mem 0xffffffffae914000-0xffffffffae9fffff] [ 3.699982] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.709407] systemd[1]: Detected virtualization kvm. [ 3.711491] systemd[1]: Detected architecture x86-64. [ 3.713496] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.740856] systemd[1]: No hostname configured. [ 3.743046] systemd[1]: Set hostname to . [ 3.745663] random: systemd: uninitialized urandom read (16 bytes read) [ 3.748415] systemd[1]: Initializing machine ID from random generator. [ 3.804124] random: ln: uninitialized urandom read (6 bytes read) [ 3.887569] random: systemd: uninitialized urandom read (16 bytes read) [ 3.890067] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.894779] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.902662] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.545627] device-mapper: uevent: version 1.0.3 [ 4.547594] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.327752] random: fast init done [ 5.356977] virtio_net virtio0 ens2: renamed from eth0 [ 5.433842] scsi host0: ata_piix [ 5.437933] scsi host1: ata_piix [ 5.440431] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.442189] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.196230] dracut-initqueue[578]: RTNETLINK answers: File exists [ 10.136515] random: crng init done [ 10.137880] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.640532] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.823680] printk: systemd: 24 output lines suppressed due to ratelimiting [ 12.105830] SELinux: Disabled at runtime. [ 12.164585] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.171262] systemd[1]: Detected virtualization kvm. [ 12.172832] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.708486] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.711777] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.718349] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.726151] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.732271] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.760923] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.774485] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ 12.950264] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 14.262630] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.389293] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.490179] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 16.238831] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 16.343958] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 21.021276] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 21.848489] NFS: Registering the id_resolver key type [ 21.856765] Key type id_resolver registered [ 21.858521] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg139-server login: [ 98.933953] libcfs: loading out-of-tree module taints kernel. [ 98.995893] Key type ._llcrypt registered [ 98.998827] Key type .llcrypt registered [ 99.130303] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 123.735561] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 125.925950] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 125.953304] alg: No test for adler32 (adler32-zlib) [ 127.799164] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 128.810567] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 130.551287] Key type lgssc registered [ 133.125127] Lustre: Echo OBD driver; http://www.lustre.org/ [ 150.639160] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 193.064177] hrtimer: interrupt took 12183111 ns [ 195.902764] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 208.792924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 208.832083] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 210.173504] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 210.205975] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 210.287866] Lustre: lustre-MDT0000: new disk, initializing [ 210.383425] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 210.398840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 216.355406] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 232.898338] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 233.084102] Lustre: 6505:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 233.155315] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 233.163674] Lustre: Skipped 1 previous similar message [ 233.352566] Lustre: lustre-MDT0001: new disk, initializing [ 233.553299] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 233.638738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 233.649137] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 238.210536] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.341430] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 253.357144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 253.794428] Lustre: lustre-OST0000: new disk, initializing [ 253.802877] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 253.814159] Lustre: 8457:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 253.909433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 260.428570] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 261.712252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 261.724672] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 261.846747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 276.190658] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 276.387216] Lustre: lustre-OST0001: new disk, initializing [ 276.392554] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 276.401221] Lustre: 9529:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 276.484165] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 283.729232] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 284.220037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 284.241183] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 284.331851] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 298.093696] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 308.237456] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 315.445596] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing check_logdir /tmp/testlogs/ [ 321.065772] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing yml_node [ 326.288457] Lustre: DEBUG MARKER: Client: 2.17.59.3 [ 329.574734] Lustre: DEBUG MARKER: MDS: 2.17.59.3 [ 332.669869] Lustre: DEBUG MARKER: OSS: 2.17.59.3 [ 334.578782] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Oct 2 09:02:45 EDT 2026 [ 353.828282] Lustre: DEBUG MARKER: only running tests: 76-120 [ 355.791948] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 357.350731] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 376.287924] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 376.290283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 376.324386] Lustre: Skipped 1 previous similar message [ 376.340775] Lustre: Skipped 3 previous similar messages [ 381.026546] Lustre: server umount lustre-MDT0000 complete [ 385.729282] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946218 with bad export cookie 82406890858593196 [ 385.739239] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 385.740581] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 386.528164] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 386.528856] LustreError: 12216:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 386.531819] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 386.543850] Lustre: Skipped 3 previous similar messages [ 386.584171] LustreError: 12216:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 390.560794] LustreError: 10080:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 390.560815] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 390.593525] LustreError: 10080:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 390.611406] Lustre: Skipped 2 previous similar messages [ 395.681707] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 395.683534] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 395.708957] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 395.714377] Lustre: Skipped 1 previous similar message [ 400.351175] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 400.785621] Lustre: server umount lustre-MDT0001 complete [ 405.430622] Lustre: server umount lustre-OST0000 complete [ 410.371700] Lustre: server umount lustre-OST0001 complete [ 418.239430] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 430.753605] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 472.288724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 472.759725] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 472.901072] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 473.091433] Lustre: lustre-MDT0000: new disk, initializing [ 473.264583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 473.288296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 478.603897] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 490.431831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 490.564585] Lustre: 15140:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 490.627788] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 490.631043] Lustre: Skipped 1 previous similar message [ 490.697480] Lustre: lustre-MDT0001: new disk, initializing [ 490.783342] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 490.823967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 490.842470] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 495.604770] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 506.431439] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 512.150445] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 519.105460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 519.536056] Lustre: lustre-OST0000: new disk, initializing [ 519.539857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 519.549209] Lustre: 17109:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 519.640120] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 521.437331] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 521.460297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 521.631454] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 527.204585] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 538.738415] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 543.771766] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 544.023852] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 549.170826] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 549.332783] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 552.419044] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 552.426894] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 552.446271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 552.483805] Lustre: Skipped 1 previous similar message [ 562.677827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 562.691950] Lustre: Skipped 4 previous similar messages [ 566.239984] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 566.427250] Lustre: server umount lustre-OST0000 complete [ 572.897203] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 572.925338] Lustre: Skipped 1 previous similar message [ 576.962083] Lustre: server umount lustre-MDT0000 complete [ 578.020105] LustreError: 15153:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 578.052085] LustreError: 15153:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 580.987791] LustreError: 15132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946413 with bad export cookie 82406890858594729 [ 581.001638] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 581.375899] Lustre: server umount lustre-MDT0001 complete [ 586.851386] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:06:57 (1790946417) [ 594.096901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 594.686948] LustreError: 19378:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 594.772201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 599.665440] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 605.157492] LustreError: 19379:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 605.185239] LustreError: 19379:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 609.270861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 614.577564] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 622.589247] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 628.919530] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 636.408956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 643.326470] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 651.455970] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 658.411423] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 666.089975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 666.109112] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 666.132545] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 666.138191] Lustre: Skipped 2 previous similar messages [ 671.957711] Lustre: server umount lustre-MDT0000 complete [ 674.792180] LustreError: 20106:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 674.806792] LustreError: 20106:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 676.072604] LustreError: 21917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946508 with bad export cookie 82406890858595611 [ 676.080714] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 676.084655] LustreError: 21917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 676.499102] Lustre: server umount lustre-MDT0001 complete [ 691.174795] Lustre: server umount lustre-OST0000 complete [ 714.224567] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 724.176591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 724.690889] LustreError: 23773:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 724.755211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 724.766351] Lustre: Skipped 2 previous similar messages [ 729.698629] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 738.907577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 744.183550] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 747.298326] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 755.612712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 763.424811] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 774.449322] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 774.667344] Lustre: lustre-OST0001: new disk, initializing [ 774.671685] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 774.681290] Lustre: 26088:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 775.984058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 775.996069] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 776.128300] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 781.181793] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 791.844929] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 796.607156] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 810.981409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 811.005224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 811.031634] Lustre: Skipped 2 previous similar messages [ 811.043839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 811.051563] Lustre: Skipped 3 previous similar messages [ 816.132968] Lustre: server umount lustre-MDT0000 complete [ 820.194356] LustreError: 23774:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 820.228844] LustreError: 23774:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 821.180991] LustreError: 23755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946654 with bad export cookie 82406890858596451 [ 821.197946] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 821.206544] LustreError: 23755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 822.173689] Lustre: server umount lustre-MDT0001 complete [ 836.680261] Lustre: server umount lustre-OST0000 complete [ 850.572715] Lustre: server umount lustre-OST0001 complete [ 868.741649] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 879.751887] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 880.280515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 880.302355] Lustre: Skipped 3 previous similar messages [ 885.622566] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 893.991453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 898.587099] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 901.551722] Lustre: 30260:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 908.659721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 915.493490] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 924.158946] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 931.038448] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 938.988714] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 942.756619] Lustre: 32139:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 950.757805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 950.763513] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 950.775247] Lustre: Skipped 3 previous similar messages [ 950.786920] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 950.795743] Lustre: Skipped 8 previous similar messages [ 954.796866] Lustre: server umount lustre-MDT0000 complete [ 956.390354] LustreError: 29846:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 956.417753] LustreError: 29846:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 959.336704] LustreError: 29092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946792 with bad export cookie 82406890858597753 [ 959.348326] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 959.371529] LustreError: 29092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 960.023507] Lustre: server umount lustre-MDT0001 complete [ 975.558540] Lustre: server umount lustre-OST0000 complete [ 977.695549] Lustre: 3640:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790946794/real 1790946794] req@ffff91d1033bf480 x1877942952307840/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790946810 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 980.938259] Lustre: 3638:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790946797/real 1790946797] req@ffff91d1033bce00 x1877942952308096/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790946813 ref 1 fl Rpc:EXNQr/200/ffffffff rc -5/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 981.144217] Lustre: server umount lustre-OST0001 complete [ 990.250836] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:13:41 (1790946821) [ 1016.993445] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1028.128135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1034.136672] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1042.990055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1048.421661] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1051.797319] Lustre: 36092:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1059.860589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1066.613764] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1075.712039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1083.620947] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1091.887891] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1094.858708] Lustre: 37972:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1101.800061] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1101.805369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1101.816086] Lustre: Skipped 6 previous similar messages [ 1101.830770] Lustre: Skipped 7 previous similar messages [ 1106.662920] Lustre: server umount lustre-MDT0000 complete [ 1111.030437] LustreError: 37282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790946943 with bad export cookie 82406890858598859 [ 1111.040574] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1111.043610] LustreError: 37282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1111.388584] Lustre: server umount lustre-MDT0001 complete [ 1125.886971] Lustre: server umount lustre-OST0000 complete [ 1127.903937] Lustre: 3641:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790946944/real 1790946944] req@ffff91d204a8a300 x1877942952357120/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790946960 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1129.674922] Lustre: server umount lustre-OST0001 complete [ 1139.309782] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:16:10 (1790946970) [ 1154.046581] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1163.892452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1164.375342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1164.380399] Lustre: Skipped 7 previous similar messages [ 1168.108838] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1178.138403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1184.492761] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1187.452549] Lustre: 41279:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1194.147383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1201.698727] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1210.856744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1218.831880] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1227.163716] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1231.333715] Lustre: 43156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1232.559379] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1232.568987] Lustre: Skipped 1 previous similar message [ 1236.976656] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1237.017189] Lustre: Skipped 4 previous similar messages [ 1238.407836] Lustre: server umount lustre-MDT0000 complete [ 1242.090723] LustreError: 40123:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1242.118921] LustreError: 40123:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 20 previous similar messages [ 1243.143874] LustreError: 40109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790947076 with bad export cookie 82406890858599958 [ 1243.151430] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1243.157112] LustreError: 40109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1249.644799] Lustre: server umount lustre-MDT0001 complete [ 1258.833307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1275.081368] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1279.507241] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1284.651255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1290.168907] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1290.233983] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1290.239214] Lustre: Skipped 1 previous similar message [ 1299.318036] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1300.998350] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1308.187591] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1310.281207] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1320.961673] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1322.713981] Lustre: server umount lustre-MDT0000 complete [ 1327.834949] LustreError: 40879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790947160 with bad export cookie 82406890858601092 [ 1327.847796] LustreError: 40879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1327.867454] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1328.275916] Lustre: server umount lustre-MDT0001 complete [ 1342.685508] Lustre: server umount lustre-OST0000 complete [ 1357.118286] Lustre: server umount lustre-OST0001 complete [ 1367.936463] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:19:58 (1790947198) [ 1385.775710] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1395.105612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1400.363803] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1410.394662] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1416.019646] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1418.784588] Lustre: 48919:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1425.933593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1433.176259] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1443.005538] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1450.009646] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1458.388484] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1462.982517] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1474.021425] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1474.025811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1474.039824] Lustre: Skipped 9 previous similar messages [ 1474.047760] Lustre: Skipped 15 previous similar messages [ 1479.148141] Lustre: server umount lustre-MDT0000 complete [ 1482.300087] LustreError: 50116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790947315 with bad export cookie 82406890858601890 [ 1482.305720] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1482.308492] LustreError: 50116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1482.519779] Lustre: server umount lustre-MDT0001 complete [ 1497.182720] Lustre: server umount lustre-OST0000 complete [ 1501.153393] Lustre: 3638:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790947317/real 1790947317] req@ffff91d10336c380 x1877942952504320/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790947333 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1502.288806] Lustre: server umount lustre-OST0001 complete [ 1512.858343] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:22:23 (1790947343) [ 1515.791979] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1517.562033] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:22:28 (1790947348) [ 1527.726790] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 1536.121321] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1560.201102] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1560.590700] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1560.666647] Lustre: lustre-MDT0000: new disk, initializing [ 1560.815171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1565.638036] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1575.085732] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1582.874725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1583.172419] Lustre: lustre-OST0000: new disk, initializing [ 1583.179182] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1583.187453] Lustre: Skipped 1 previous similar message [ 1583.192386] Lustre: 54994:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1585.366187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1585.385176] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1585.416827] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1589.897741] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1600.057951] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1602.534523] Lustre: DEBUG MARKER: create test files [ 1641.955841] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1645.550066] Lustre: server umount lustre-OST0000 complete [ 1649.726845] Lustre: server umount lustre-MDT0000 complete [ 1660.560242] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 1663.372733] Key type lgssc unregistered [ 1663.718733] LNet: 56622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1663.723767] LNetError: 56622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1663.737330] LNet: Removed LNI 192.168.201.139@tcp [ 1664.663178] Key type .llcrypt unregistered [ 1664.669087] Key type ._llcrypt unregistered [ 1695.299347] Key type ._llcrypt registered [ 1695.305142] Key type .llcrypt registered [ 1695.422749] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1696.992442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1697.045894] alg: No test for adler32 (adler32-zlib) [ 1698.303461] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 1698.603586] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 1700.336001] Key type lgssc registered [ 1701.753520] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1710.699470] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1710.749845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1712.387932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1716.819518] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1726.858501] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1734.739275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1735.390238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1737.504187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1737.723313] Lustre: 59380:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1737.730029] Lustre: 59380:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1737.734710] Lustre: 59380:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1737.746756] Lustre: 59380:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1737.753466] Lustre: 59380:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1737.767605] Lustre: 59380:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1742.115742] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1750.380343] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1755.582413] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1761.503597] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1769.065531] Lustre: server umount lustre-OST0000 complete [ 1782.464395] Lustre: server umount lustre-MDT0000 complete [ 1794.420739] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 1797.207020] Key type lgssc unregistered [ 1797.611146] LNet: 60986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1797.618220] LNetError: 60986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1798.629816] LNet: Removed LNI 192.168.201.139@tcp [ 1799.621383] Key type .llcrypt unregistered [ 1799.624104] Key type ._llcrypt unregistered [ 1831.377151] Key type ._llcrypt registered [ 1831.380481] Key type .llcrypt registered [ 1831.460310] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1833.120967] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1833.160610] alg: No test for adler32 (adler32-zlib) [ 1834.366242] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 1834.664343] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 1836.407243] Key type lgssc registered [ 1837.859245] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1846.599924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1846.636336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1848.115293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1853.019767] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1861.472633] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1869.391379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1869.936402] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.011405] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1873.103329] Lustre: 63843:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1873.112704] Lustre: 63843:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1873.120626] Lustre: 63843:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1873.130373] Lustre: 63843:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1873.142315] Lustre: 63843:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1873.150936] Lustre: 63843:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1876.178879] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 1884.435770] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1890.721862] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1898.976712] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1898.992905] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1899.018045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1904.098236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1904.581778] Lustre: server umount lustre-OST0000 complete [ 1909.904614] Lustre: server umount lustre-MDT0000 complete [ 1923.312916] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 1925.947657] Key type lgssc unregistered [ 1926.217937] LNet: 65400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1926.229395] LNetError: 65400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1926.251760] LNet: Removed LNI 192.168.201.139@tcp [ 1927.144263] Key type .llcrypt unregistered [ 1927.148054] Key type ._llcrypt unregistered [ 1944.817386] Key type ._llcrypt registered [ 1944.823504] Key type .llcrypt registered [ 1944.926637] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 1959.711738] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 1961.341232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1961.357520] alg: No test for adler32 (adler32-zlib) [ 1962.554930] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 1962.891157] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 1964.623211] Key type lgssc registered [ 1966.582899] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2005.680633] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2005.719393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2006.952180] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2006.999287] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2007.080867] Lustre: lustre-MDT0000: new disk, initializing [ 2007.250613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2007.274619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2012.280470] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2022.730608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2022.835558] Lustre: 68570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2022.894154] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2022.898121] Lustre: Skipped 1 previous similar message [ 2022.989315] Lustre: lustre-MDT0001: new disk, initializing [ 2023.075849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2023.103935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2023.118212] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2027.885653] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2037.174479] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2044.008637] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2051.041214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2051.316766] Lustre: lustre-OST0000: new disk, initializing [ 2051.319997] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2051.328297] Lustre: 70539:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2051.430282] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2057.541217] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2061.366462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2061.376204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2061.451981] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2067.358662] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2072.561959] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2072.769344] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2077.866822] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2078.084785] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2082.276594] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2082.280091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2082.288750] Lustre: Skipped 1 previous similar message [ 2082.296889] Lustre: Skipped 1 previous similar message [ 2086.461645] Lustre: server umount lustre-OST0000 complete [ 2092.522236] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2092.524344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2092.545272] Lustre: Skipped 1 previous similar message [ 2096.756306] Lustre: server umount lustre-MDT0000 complete [ 2101.362639] LustreError: 69364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790947934 with bad export cookie 9970736058805685474 [ 2101.369280] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2101.384751] LustreError: 69364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2101.927987] Lustre: server umount lustre-MDT0001 complete [ 2110.990250] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:32:21 (1790947941) [ 2122.939624] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 2164.627766] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 2175.249532] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 2210.829354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2211.108672] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2211.157902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2211.274251] Lustre: lustre-MDT0000: new disk, initializing [ 2211.378662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2211.397123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2216.554187] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2227.757811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2227.917390] Lustre: 76555:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2227.982789] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2227.988568] Lustre: Skipped 1 previous similar message [ 2228.087816] Lustre: lustre-MDT0001: new disk, initializing [ 2228.246406] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2228.305422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2228.321874] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2232.649151] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2242.109853] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2249.127871] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2256.072164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2256.315496] Lustre: lustre-OST0000: new disk, initializing [ 2256.320336] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2256.326292] Lustre: 78524:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2256.389964] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.572587] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2265.191408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2265.200807] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2265.318900] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2271.656884] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2276.411731] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2276.613568] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2281.111650] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2281.371797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2286.066090] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2286.082949] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2286.094461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.103738] Lustre: Skipped 1 previous similar message [ 2286.120750] Lustre: Skipped 2 previous similar messages [ 2291.168566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2291.188467] Lustre: Skipped 1 previous similar message [ 2296.292178] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2296.307041] Lustre: Skipped 1 previous similar message [ 2297.311307] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2297.592649] Lustre: server umount lustre-OST0000 complete [ 2304.992607] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2305.003896] LustreError: Skipped 1 previous similar message [ 2305.012937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2305.037081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2308.683528] Lustre: server umount lustre-MDT0000 complete [ 2311.649637] LustreError: 76569:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2311.677243] LustreError: 76569:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2312.482782] LustreError: 76548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790948145 with bad export cookie 9970736058805686363 [ 2312.501921] LustreError: 76548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2312.503170] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2312.954887] Lustre: server umount lustre-MDT0001 complete [ 2321.877113] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:35:52 (1790948152) [ 2328.720105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2329.279950] LustreError: 80939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2329.373058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2334.697915] LustreError: 80940:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2334.843444] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2339.828358] LustreError: 80939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2344.883466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2344.944864] LustreError: 80940:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2351.072634] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2359.380583] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2365.238820] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2372.993886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2373.402983] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2373.409661] Lustre: Skipped 1 previous similar message [ 2379.688711] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2387.188221] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2391.067554] Lustre: 83611:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 7e86a24c-16ae-4477-a9fe-5097148bc6c4 at adminstrative request [ 2396.135783] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2396.146991] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2406.370595] Lustre: MGS: Client 7e86a24c-16ae-4477-a9fe-5097148bc6c4 (at 0@lo) reconnecting [ 2406.378855] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8a5f2b9e464a93cd to 0x8a5f2b9e464a966d [ 2406.395061] Lustre: MGC192.168.201.139@tcp: Connection restored to 0@lo (at 0@lo) [ 2428.470436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2428.647774] Lustre: lustre-OST0001: new disk, initializing [ 2428.651550] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2428.662798] Lustre: 84005:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2434.573428] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2438.254251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2438.269936] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2438.359656] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2445.169736] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2448.865358] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2448.875887] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2448.878976] LustreError: Skipped 1 previous similar message [ 2448.887991] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2448.892967] Lustre: Skipped 2 previous similar messages [ 2448.909534] Lustre: Skipped 3 previous similar messages [ 2454.622938] Lustre: server umount lustre-OST0001 complete [ 2459.118301] LustreError: 82760:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2459.138584] LustreError: 82760:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2461.667259] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2461.674709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2465.742761] Lustre: server umount lustre-OST0000 complete [ 2473.442694] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2473.460877] LustreError: Skipped 1 previous similar message [ 2476.389632] Lustre: server umount lustre-MDT0000 complete [ 2479.585179] LustreError: 80940:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2479.599915] LustreError: 80940:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2480.195681] LustreError: 84003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790948313 with bad export cookie 9970736058805687917 [ 2480.202492] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2480.213523] LustreError: 84003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2480.495967] Lustre: server umount lustre-MDT0001 complete [ 2489.676862] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:38:40 (1790948320) [ 2491.146168] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2492.794328] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:38:43 (1790948323) [ 2494.207081] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2496.061938] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:38:46 (1790948326) [ 2497.376227] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2499.097937] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:38:50 (1790948330) [ 2500.698853] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2502.539633] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:38:53 (1790948333) [ 2503.942888] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2509.909326] loop: module loaded [ 2509.945385] loop0: detected capacity change from 0 to 4294967296 [ 2510.047385] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2511.453789] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2516.433476] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2520.739950] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2548.158295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2548.221051] LustreError: 88126:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2548.232600] LustreError: 88126:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2548.237520] LustreError: 88126:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2548.247399] LustreError: 88126:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2569.354833] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 2579.465825] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 2612.200447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2612.394500] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2612.410323] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2612.467899] Lustre: lustre-MDT0000: new disk, initializing [ 2612.517495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2612.527537] Lustre: Skipped 1 previous similar message [ 2612.541517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2617.418178] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2628.059648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2628.239095] Lustre: 90991:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2628.545724] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2632.380110] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2639.985521] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2645.498142] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2651.257433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2651.556828] Lustre: 92966:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2657.743759] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2659.425595] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2659.524260] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2666.979733] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2671.029921] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2671.204933] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2675.535309] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2675.741850] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2680.297111] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2680.297721] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2680.311277] Lustre: Skipped 3 previous similar messages [ 2680.317447] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2680.317454] Lustre: Skipped 9 previous similar messages [ 2680.344463] LustreError: Skipped 1 previous similar message [ 2683.700252] Lustre: server umount lustre-OST0000 complete [ 2693.649354] Lustre: server umount lustre-MDT0000 complete [ 2695.649406] LustreError: 92761:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2695.683348] LustreError: 92761:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2696.597181] LustreError: 91790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790948529 with bad export cookie 9970736058805688533 [ 2696.600676] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2696.606756] LustreError: 91790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2696.868274] Lustre: server umount lustre-MDT0001 complete [ 2703.606538] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:42:14 (1790948534) [ 2709.038242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2714.115739] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2722.495455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2727.952348] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2734.674879] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2739.872170] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2747.442771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2747.819921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2747.824309] Lustre: Skipped 4 previous similar messages [ 2753.575096] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2759.864339] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2764.116452] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2764.328656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2768.695563] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2768.855468] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2774.801566] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2774.962248] Lustre: lustre-OST0001: new disk, initializing [ 2774.964819] Lustre: Skipped 2 previous similar messages [ 2774.976417] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2774.983121] Lustre: Skipped 3 previous similar messages [ 2774.989626] Lustre: 98744:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2780.702432] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2782.848558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2782.863827] Lustre: Skipped 2 previous similar messages [ 2782.870805] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2782.946405] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2789.118087] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2792.180777] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2792.356698] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2796.126947] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2796.300255] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2814.842972] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2838.731152] Lustre: Failing over lustre-MDT0000 [ 2839.407835] LustreError: 99037:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2839.425127] LustreError: 99037:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2839.533904] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2839.561025] Lustre: Skipped 6 previous similar messages [ 2841.219262] Lustre: server umount lustre-MDT0000 complete [ 2855.904355] Lustre: 66784:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790948672/real 1790948672] req@00000000ade010ea x1877944875356544/t0(0) o400->MGC192.168.201.139@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790948688 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2855.935313] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2866.781207] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2866.785780] LDISKFS-fs (dm-0): recovery complete [ 2866.794805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2881.524652] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x8a5f2b9e464b60e8 [ 2881.544175] Lustre: MGC192.168.201.139@tcp: Connection restored to 0@lo (at 0@lo) [ 2881.999685] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2886.490262] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2886.497148] Lustre: 99037:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2886.871185] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 2887.140133] Lustre: 96105:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2887.147069] Lustre: 96105:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2887.153722] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2887.233585] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2887.586733] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2887.915034] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2887.926097] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2888.247069] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2888.252129] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2888.261374] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2888.271739] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2888.914697] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2888.928932] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2889.567283] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2889.575431] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 3 previous similar messages [ 2890.552173] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2890.571506] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2891.204979] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2891.221843] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2891.855157] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2891.857773] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2894.712448] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2894.735984] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2895.376358] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2895.389856] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2896.039486] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2896.046444] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2902.728896] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2902.745303] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2903.397911] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2903.411178] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2904.042870] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2904.055628] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2918.920323] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2918.932755] Lustre: 101599:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2919.568252] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2919.580261] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2920.207593] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2920.221833] LustreError: 101599:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2946.191227] Lustre: 101599:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2946.204192] Lustre: 101599:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2946.212815] Lustre: 101599:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 04982d97-91e7-4cf5-bb19-b6d80f677579@192.168.201.39@tcp [ 2946.230816] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2946.240490] Lustre: 101599:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@00000000a3ec469f x1877944859433856/t0(8589934782) o35->04982d97-91e7-4cf5-bb19-b6d80f677579@192.168.201.39@tcp:454/0 lens 392/0 e 0 to 0 dl 1790948789 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2946.280465] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2946.292697] Lustre: lustre-MDT0000: Denying connection for new client 04982d97-91e7-4cf5-bb19-b6d80f677579 (at 192.168.201.39@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2946.317991] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2946.348500] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2946.354062] Lustre: Skipped 2 previous similar messages [ 2946.365868] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2946.406916] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2946.413985] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 2958.820253] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2958.835690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2958.837033] Lustre: Skipped 1 previous similar message [ 2958.841178] Lustre: Skipped 3 previous similar messages [ 2964.662321] Lustre: server umount lustre-OST0000 complete [ 2969.064656] LustreError: 101976:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2969.095048] LustreError: 101976:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 59 previous similar messages [ 2974.647441] Lustre: server umount lustre-OST0001 complete [ 2984.683543] Lustre: server umount lustre-MDT0000 complete [ 2988.384216] LustreError: 95356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790948821 with bad export cookie 9970736058805739752 [ 2988.386068] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2988.392732] LustreError: 95356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2988.976552] Lustre: server umount lustre-MDT0001 complete [ 2996.384583] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:47:07 (1790948827) [ 3003.559987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3003.575722] Lustre: *** cfs_fail_loc=197, val=0*** [ 3019.169272] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3024.287921] LustreError: 103572:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.139@tcp: failed processing log, type 4: rc = -110 [ 3029.426459] Lustre: *** cfs_fail_loc=197, val=0*** [ 3029.430600] Lustre: Skipped 65 previous similar messages [ 3049.951296] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3049.960922] Lustre: Skipped 2 previous similar messages [ 3055.422275] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3061.935547] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3064.047853] Lustre: server umount lustre-OST0000 complete [ 3071.538887] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:48:22 (1790948902) [ 3093.866432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3099.745238] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3109.882299] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3114.172551] LustreError: 105225:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3114.186825] LustreError: 105225:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3114.199672] LustreError: 105225:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3114.838084] Lustre: server umount lustre-MDT0000 complete [ 3153.638964] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 3164.959627] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 3202.471773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3202.663430] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3202.694983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3202.759480] Lustre: lustre-MDT0000: new disk, initializing [ 3202.828243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3207.673906] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3219.817960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3219.998797] Lustre: 109404:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3220.260916] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3224.796181] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3235.197787] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3241.194791] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3248.170561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3248.484353] Lustre: 111373:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3254.520369] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3256.457770] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3263.147485] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3268.028754] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3268.296452] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3272.613860] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3272.828798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3276.779680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3276.782075] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3276.794628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3276.794637] Lustre: Skipped 10 previous similar messages [ 3276.802280] Lustre: Skipped 5 previous similar messages [ 3276.805507] LustreError: Skipped 1 previous similar message [ 3281.193453] Lustre: server umount lustre-OST0000 complete [ 3291.987372] Lustre: server umount lustre-MDT0000 complete [ 3292.141186] LustreError: 110906:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3292.186744] LustreError: 110906:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 3296.356427] LustreError: 109395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790949129 with bad export cookie 9970736058805750476 [ 3296.359067] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3296.367981] LustreError: 109395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3296.901461] Lustre: server umount lustre-MDT0001 complete [ 3306.302961] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:52:17 (1790949137) [ 3316.695360] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 3319.456808] Key type lgssc unregistered [ 3319.779790] LNet: 113860:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3319.788434] LNetError: 113860:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3320.809433] LNet: Removed LNI 192.168.201.139@tcp [ 3321.913196] Key type .llcrypt unregistered [ 3321.914610] Key type ._llcrypt unregistered [ 3341.477262] Key type ._llcrypt registered [ 3341.480913] Key type .llcrypt registered [ 3341.570893] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 3360.563953] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 3362.420888] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3362.454209] alg: No test for adler32 (adler32-zlib) [ 3363.666845] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 3364.120252] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 3365.874561] Key type lgssc registered [ 3366.712757] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3406.121503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3406.150587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3407.342581] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3407.370349] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3407.452274] Lustre: lustre-MDT0000: new disk, initializing [ 3407.562856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3407.576299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3411.628366] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3421.474550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3421.737257] Lustre: lustre-OST0000: new disk, initializing [ 3421.740420] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3421.754515] Lustre: Skipped 1 previous similar message [ 3421.766576] Lustre: 118046:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3421.849527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3428.868632] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3431.536460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3431.544884] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3431.589872] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3440.741906] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3440.963235] Lustre: lustre-OST0001: new disk, initializing [ 3440.967446] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3440.973766] Lustre: 118909:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3441.058358] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3447.502836] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3449.525773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3449.534413] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3449.559477] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3460.067124] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3460.079846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3463.674295] Lustre: server umount lustre-MDT0000 complete [ 3469.732860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3474.329276] LustreError: 117234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790949307 with bad export cookie 2455000633111482064 [ 3474.331207] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3474.339988] LustreError: 117234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3474.470757] Lustre: server umount lustre-OST0000 complete [ 3477.960076] Lustre: server umount lustre-OST0001 complete [ 3487.235162] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:55:18 (1790949318) [ 3507.977888] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:55:38 (1790949338) [ 3517.266872] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 3538.305756] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:56:09 (1790949369) [ 3573.645127] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 3582.650197] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 3614.969974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3615.184751] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3615.205137] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3615.262339] Lustre: lustre-MDT0000: new disk, initializing [ 3615.313218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3615.324327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3619.802628] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3629.322127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3629.448479] Lustre: 126664:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3629.521409] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3629.524500] Lustre: Skipped 1 previous similar message [ 3629.657428] Lustre: lustre-MDT0001: new disk, initializing [ 3629.772042] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3629.827377] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3629.842308] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3634.608573] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3645.145330] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3651.405891] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3658.420617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3658.595256] Lustre: lustre-OST0000: new disk, initializing [ 3658.599875] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3658.605415] Lustre: 128632:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3658.685364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3663.154493] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3667.552119] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3667.558075] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3667.672947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3671.795424] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3676.769686] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3677.060475] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3681.380266] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3681.573993] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3688.420680] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3688.421221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3688.425735] LustreError: Skipped 1 previous similar message [ 3688.429280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3688.429292] Lustre: Skipped 1 previous similar message [ 3688.465496] Lustre: Skipped 2 previous similar messages [ 3693.542601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3693.545675] Lustre: Skipped 2 previous similar messages [ 3698.146196] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3698.419048] Lustre: server umount lustre-OST0000 complete [ 3701.728707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3701.733192] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3701.753365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3703.779874] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3707.904296] Lustre: server umount lustre-MDT0000 complete [ 3711.845921] LustreError: 126656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790949544 with bad export cookie 2455000633111483282 [ 3711.851947] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3711.856489] LustreError: 126656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3712.080894] Lustre: server umount lustre-MDT0001 complete [ 3719.932829] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:59:11 (1790949551) [ 3725.835020] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3726.272152] LustreError: 131047:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3726.290675] LustreError: 131047:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3726.360403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3730.753827] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3731.426669] LustreError: 131048:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3736.560055] LustreError: 131047:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3739.238267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3744.020932] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3751.548754] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3757.368439] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3763.762441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3764.011409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3764.015437] Lustre: Skipped 1 previous similar message [ 3769.913536] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3776.541334] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3780.451457] Lustre: *** cfs_fail_loc=159, val=0*** [ 3795.844127] Lustre: lustre-MDT0000: Client cec1d9b5-cdab-4bf3-8193-e40c76e67eac (at 192.168.201.39@tcp) reconnecting [ 3797.215766] Lustre: *** cfs_fail_loc=159, val=0*** [ 3797.223199] Lustre: Skipped 5 previous similar messages [ 3812.751975] Lustre: lustre-MDT0000: Client cec1d9b5-cdab-4bf3-8193-e40c76e67eac (at 192.168.201.39@tcp) reconnecting [ 3831.778711] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3831.791286] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3831.821702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3831.830204] Lustre: Skipped 2 previous similar messages [ 3836.975602] Lustre: server umount lustre-OST0000 complete [ 3842.016688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3842.042186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3842.069182] Lustre: Skipped 1 previous similar message [ 3842.075200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3842.087898] Lustre: Skipped 2 previous similar messages [ 3846.830785] Lustre: server umount lustre-MDT0000 complete [ 3848.162146] LustreError: 133705:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3848.187750] LustreError: 133705:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3850.672502] LustreError: 131028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790949683 with bad export cookie 2455000633111484171 [ 3850.680195] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3850.685864] LustreError: 131028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3851.088028] Lustre: server umount lustre-MDT0001 complete [ 3863.113548] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 3866.670271] Key type lgssc unregistered [ 3866.960689] LNet: 134837:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3866.968348] LNetError: 134837:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3866.977858] LNet: Removed LNI 192.168.201.139@tcp [ 3867.944697] Key type .llcrypt unregistered [ 3867.947481] Key type ._llcrypt unregistered [ 3894.848867] Key type ._llcrypt registered [ 3894.851640] Key type .llcrypt registered [ 3894.962112] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 3895.731748] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3895.744124] alg: No test for adler32 (adler32-zlib) [ 3896.759969] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 3897.053664] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 3898.759236] Key type lgssc registered [ 3900.604812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3910.386615] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3910.423539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3912.200453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3917.755657] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3922.404193] LustreError: 136269:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3922.425288] LustreError: 136269:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3927.181767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.655778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3931.485345] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3939.268033] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3946.060191] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3953.259597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3953.631450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3957.764477] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3959.700899] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 3966.248943] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3978.787150] Lustre: *** cfs_fail_loc=159, val=0*** [ 3995.043583] Lustre: lustre-MDT0000: Client f3a9013b-8364-4b16-90ad-278c17d16cdd (at 192.168.201.39@tcp) reconnecting [ 3997.228535] Lustre: *** cfs_fail_loc=159, val=0*** [ 4013.476774] Lustre: lustre-MDT0001: Client f3a9013b-8364-4b16-90ad-278c17d16cdd (at 192.168.201.39@tcp) reconnecting [ 4015.205704] Lustre: *** cfs_fail_loc=159, val=0*** [ 4015.207454] Lustre: Skipped 3 previous similar messages [ 4030.892407] Lustre: lustre-MDT0001: Client f3a9013b-8364-4b16-90ad-278c17d16cdd (at 192.168.201.39@tcp) reconnecting [ 4036.979230] Lustre: 139307:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4039.072638] Lustre: *** cfs_fail_loc=159, val=0*** [ 4039.087029] Lustre: Skipped 3 previous similar messages [ 4054.898586] Lustre: lustre-MDT0000: Client c74719fc-977c-4ad7-857c-a7f06a75c486 (at 192.168.201.39@tcp) reconnecting [ 4057.065218] Lustre: *** cfs_fail_loc=159, val=0*** [ 4057.075805] Lustre: Skipped 14 previous similar messages [ 4072.896282] Lustre: lustre-MDT0000: Client c74719fc-977c-4ad7-857c-a7f06a75c486 (at 192.168.201.39@tcp) reconnecting [ 4074.757846] Lustre: 139602:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4080.609715] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4080.611814] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4080.621356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4080.632110] Lustre: Skipped 1 previous similar message [ 4080.642596] Lustre: Skipped 1 previous similar message [ 4085.729446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4086.153979] Lustre: server umount lustre-OST0000 complete [ 4090.848614] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4090.850192] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4090.868579] Lustre: Skipped 1 previous similar message [ 4090.890171] Lustre: Skipped 2 previous similar messages [ 4095.575090] Lustre: server umount lustre-MDT0000 complete [ 4095.972569] LustreError: 139366:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4095.990222] LustreError: 139366:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4098.664336] LustreError: 136251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790949931 with bad export cookie 15775476199334631252 [ 4098.670035] LustreError: 136251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4098.670283] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4098.977862] Lustre: server umount lustre-MDT0001 complete [ 4109.554990] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4112.062127] Key type lgssc unregistered [ 4112.379203] LNet: 140653:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4112.386829] LNetError: 140653:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4112.397843] LNet: Removed LNI 192.168.201.139@tcp [ 4113.295747] Key type .llcrypt unregistered [ 4113.300781] Key type ._llcrypt unregistered [ 4136.958579] Key type ._llcrypt registered [ 4136.961179] Key type .llcrypt registered [ 4137.057350] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4138.660839] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4138.675578] alg: No test for adler32 (adler32-zlib) [ 4139.710798] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 4140.078525] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 4141.799262] Key type lgssc registered [ 4143.495408] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4152.179622] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4152.210540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4153.649503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4158.284629] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4164.067869] LustreError: 142082:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4164.097620] LustreError: 142082:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4167.573298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4167.809606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4171.895823] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4178.368471] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4183.402393] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4189.637754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4189.891571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4193.026845] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4196.583684] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4199.207580] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4203.461659] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4208.862774] Lustre: 144666:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4211.830749] Lustre: 144715:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4213.187698] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4213.862405] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4216.524355] Lustre: 144861:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4217.494923] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4218.537395] Lustre: 144959:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4223.971070] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4223.994068] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4224.018398] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4224.999981] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4225.001249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4225.002168] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4228.094549] Lustre: server umount lustre-OST0000 complete [ 4234.209123] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4234.215376] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4234.235602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4234.250152] Lustre: Skipped 1 previous similar message [ 4240.151894] Lustre: server umount lustre-MDT0000 complete [ 4240.359818] LustreError: 142082:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4240.412991] LustreError: 142082:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4244.475113] LustreError: 145068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790950077 with bad export cookie 2746348718388483518 [ 4244.481125] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4244.482743] LustreError: 145068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4245.016206] Lustre: server umount lustre-MDT0001 complete [ 4254.836410] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4257.577448] Key type lgssc unregistered [ 4257.902984] LNet: 146020:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4257.912617] LNetError: 146020:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4257.925513] LNet: Removed LNI 192.168.201.139@tcp [ 4258.670259] Key type .llcrypt unregistered [ 4258.672306] Key type ._llcrypt unregistered [ 4280.781945] Key type ._llcrypt registered [ 4280.786470] Key type .llcrypt registered [ 4280.922204] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4282.142865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4282.164746] alg: No test for adler32 (adler32-zlib) [ 4283.438910] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 4283.724665] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 4285.423192] Key type lgssc registered [ 4286.590674] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4293.705572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4293.726765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4295.116334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4299.317522] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4304.356100] LustreError: 147450:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4304.373394] LustreError: 147450:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4306.778259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4307.030331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4311.330727] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4318.667887] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4324.988776] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4332.431688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4332.812279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4338.725981] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4339.960462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4346.294760] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4352.242488] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4355.439139] Lustre: *** cfs_fail_loc=159, val=0*** [ 4370.831907] Lustre: lustre-MDT0000: Client cadab149-4ac5-4848-8439-988a529a808f (at 192.168.201.39@tcp) reconnecting [ 4388.320076] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4388.326559] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4388.331329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4388.346277] Lustre: Skipped 1 previous similar message [ 4388.367813] Lustre: Skipped 1 previous similar message [ 4393.442750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4393.451157] Lustre: Skipped 1 previous similar message [ 4394.367580] Lustre: server umount lustre-OST0000 complete [ 4398.567371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4398.597092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4398.602702] Lustre: Skipped 1 previous similar message [ 4403.680457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4403.688771] Lustre: Skipped 1 previous similar message [ 4408.802727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4408.816049] Lustre: Skipped 1 previous similar message [ 4412.383351] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4412.944080] Lustre: server umount lustre-MDT0000 complete [ 4413.921514] LustreError: 147445:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4413.947575] LustreError: 147445:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4417.688830] LustreError: 147429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790950250 with bad export cookie 3796835582156351344 [ 4417.701828] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4417.705812] LustreError: 147429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4418.129333] Lustre: server umount lustre-MDT0001 complete [ 4429.354080] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4432.226416] Key type lgssc unregistered [ 4432.521723] LNet: 151116:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4432.524993] LNetError: 151116:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4432.534897] LNet: Removed LNI 192.168.201.139@tcp [ 4433.328769] Key type .llcrypt unregistered [ 4433.331271] Key type ._llcrypt unregistered [ 4457.420525] Key type ._llcrypt registered [ 4457.422148] Key type .llcrypt registered [ 4457.527334] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4458.515686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4458.562571] alg: No test for adler32 (adler32-zlib) [ 4459.540371] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 4459.820251] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 4461.527228] Key type lgssc registered [ 4462.539102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4469.350601] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4469.372202] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4470.705106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4474.250709] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4480.996604] LustreError: 152562:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4481.013077] LustreError: 152562:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4483.074029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4483.453777] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4488.633373] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4495.561687] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4500.537877] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4507.172351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4507.484734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4508.552875] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4514.035169] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4515.785275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4521.652394] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4540.779164] Lustre: lustre-MDT0001: evicting b5e8fbe5-5a23-48d7-9a62-c81f218d6b0f (at 192.168.201.39@tcp) by administrative request [ 4540.786431] Lustre: Skipped 1 previous similar message [ 4570.080618] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4570.095160] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4570.110284] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4572.131196] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4572.140680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4572.153926] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4572.165823] Lustre: Skipped 1 previous similar message [ 4575.915298] Lustre: server umount lustre-OST0000 complete [ 4580.832689] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4580.851524] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4580.871487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4585.735380] Lustre: server umount lustre-MDT0000 complete [ 4587.494508] LustreError: 152562:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4587.525695] LustreError: 152562:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4588.812955] LustreError: 152542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790950421 with bad export cookie 6106192780851654931 [ 4588.819285] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4588.830068] LustreError: 152542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4589.084624] Lustre: server umount lustre-MDT0001 complete [ 4598.193450] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4600.662703] Key type lgssc unregistered [ 4600.921162] LNet: 156633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4600.934325] LNetError: 156633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4601.959163] LNet: Removed LNI 192.168.201.139@tcp [ 4602.809083] Key type .llcrypt unregistered [ 4602.814960] Key type ._llcrypt unregistered [ 4673.932555] Key type ._llcrypt registered [ 4673.937759] Key type .llcrypt registered [ 4674.067263] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4675.326969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4675.400155] alg: No test for adler32 (adler32-zlib) [ 4676.443492] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 4676.742575] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 4678.439241] Key type lgssc registered [ 4679.568575] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4686.317120] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4686.343184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4687.769498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4692.353237] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4698.087440] LustreError: 159138:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4698.105402] LustreError: 159138:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4699.136306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4699.515765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4703.317914] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4709.229308] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4715.101580] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4721.306861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4721.692603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4724.795466] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4727.076783] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4732.986925] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4734.071636] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4743.137705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4743.171328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4748.325703] Lustre: server umount lustre-OST0000 complete [ 4753.382174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.386107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4753.396729] Lustre: Skipped 2 previous similar messages [ 4753.411721] Lustre: Skipped 2 previous similar messages [ 4758.496653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4758.505441] Lustre: Skipped 1 previous similar message [ 4763.618265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4763.626628] Lustre: Skipped 1 previous similar message [ 4766.175182] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4766.598057] Lustre: server umount lustre-MDT0000 complete [ 4768.739255] LustreError: 159138:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4768.768986] LustreError: 159138:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4770.484131] LustreError: 159117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790950603 with bad export cookie 14773858478628951866 [ 4770.493435] LustreError: 159117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4770.495781] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4771.186636] Lustre: server umount lustre-MDT0001 complete [ 4782.117394] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4784.472292] Key type lgssc unregistered [ 4784.730040] LNet: 162679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4784.743416] LNetError: 162679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4784.756692] LNet: Removed LNI 192.168.201.139@tcp [ 4785.625142] Key type .llcrypt unregistered [ 4785.627458] Key type ._llcrypt unregistered [ 4806.432843] Key type ._llcrypt registered [ 4806.436994] Key type .llcrypt registered [ 4806.541948] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4807.428051] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4807.468897] alg: No test for adler32 (adler32-zlib) [ 4808.525707] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 4808.752990] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 4810.407598] Key type lgssc registered [ 4811.699143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4825.800172] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:17:36 (1790950656) [ 4847.006361] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 4856.312636] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 4891.861341] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4891.888769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4893.154934] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4893.190534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4893.260645] Lustre: lustre-MDT0000: new disk, initializing [ 4893.339303] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4893.356405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4897.576348] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4907.963328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4908.069832] Lustre: 166941:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4908.113969] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4908.119544] Lustre: Skipped 1 previous similar message [ 4908.201311] Lustre: lustre-MDT0001: new disk, initializing [ 4908.299239] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4908.342711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4908.353593] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4913.003684] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4921.870223] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4927.885571] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4934.895084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4935.258484] Lustre: lustre-OST0000: new disk, initializing [ 4935.262824] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4935.275975] Lustre: 168908:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4935.372752] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4943.218082] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 4943.433735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4943.442781] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4943.583796] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4952.229618] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4964.323105] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4964.324965] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4964.328470] LustreError: Skipped 1 previous similar message [ 4964.334017] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4964.342655] Lustre: Skipped 1 previous similar message [ 4964.367629] Lustre: Skipped 1 previous similar message [ 4968.318367] Lustre: server umount lustre-OST0000 complete [ 4974.560890] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4974.565931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4974.583563] Lustre: Skipped 1 previous similar message [ 4974.601564] Lustre: Skipped 1 previous similar message [ 4978.640056] Lustre: server umount lustre-MDT0000 complete [ 4982.220343] LustreError: 166930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790950815 with bad export cookie 5449427191674801307 [ 4982.224736] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4982.231447] LustreError: 166930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4982.535199] Lustre: server umount lustre-MDT0001 complete [ 4994.068447] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 4997.129802] Key type lgssc unregistered [ 4997.462226] LNet: 170811:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4997.467716] LNetError: 170811:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4997.484699] LNet: Removed LNI 192.168.201.139@tcp [ 4998.381780] Key type .llcrypt unregistered [ 4998.384445] Key type ._llcrypt unregistered [ 5022.887480] Key type ._llcrypt registered [ 5022.890485] Key type .llcrypt registered [ 5023.034342] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 5024.195817] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5024.347923] alg: No test for adler32 (adler32-zlib) [ 5025.483826] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 5025.722211] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 5027.375218] Key type lgssc registered [ 5028.489334] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5036.263721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5036.295220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5037.854902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5042.341825] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 5047.267098] LustreError: 172245:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5047.289072] LustreError: 172245:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5051.014522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5051.607674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5056.315194] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 5063.657711] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5069.215474] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5074.784910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5075.090544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5080.755850] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 5083.319907] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5088.544484] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5095.838816] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5095.855488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5095.870317] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5095.876953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.898777] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5097.121259] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5097.143801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5097.162024] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5097.168174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.177280] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5098.185897] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5098.209863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5098.220892] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5098.233521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.245196] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5099.158829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.949617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.664674] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5100.670783] Lustre: Skipped 2 previous similar messages [ 5100.686342] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5100.700815] Lustre: Skipped 2 previous similar messages [ 5100.706268] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5100.716816] Lustre: Skipped 2 previous similar messages [ 5100.724816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.738208] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5100.755649] Lustre: Skipped 2 previous similar messages [ 5101.572851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.435068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.280423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.075562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.021345] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5105.027932] Lustre: Skipped 4 previous similar messages [ 5105.051948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5105.061955] Lustre: Skipped 4 previous similar messages [ 5105.065715] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5105.071189] Lustre: Skipped 4 previous similar messages [ 5105.075644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.085863] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5105.092493] Lustre: Skipped 4 previous similar messages [ 5105.906458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.797813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.751169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.604648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.622608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.543452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.373808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.310361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.251387] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5113.256330] Lustre: Skipped 8 previous similar messages [ 5113.266772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5113.271921] Lustre: Skipped 8 previous similar messages [ 5113.274811] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5113.279434] Lustre: Skipped 8 previous similar messages [ 5113.282673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.295597] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5113.300510] Lustre: Skipped 8 previous similar messages [ 5114.360290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.287840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.253929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.264594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.418225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.558794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.818108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.932971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.992678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.865865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.680249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.513639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.302689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.110930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.048788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.884403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.701844] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5129.708687] Lustre: Skipped 16 previous similar messages [ 5129.728431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5129.739359] Lustre: Skipped 16 previous similar messages [ 5129.742954] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5129.748508] Lustre: Skipped 16 previous similar messages [ 5129.754331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.766554] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5129.769631] Lustre: Skipped 16 previous similar messages [ 5130.563204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.475799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.217933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.158354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.108097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.039495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.005771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.867081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.728001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.742812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.667031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.828950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.838647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.763724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.724481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.825684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.866632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.890881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.965879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.063432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.879461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.779499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.597917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.865828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.688880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.628634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.463166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.315845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.246721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.075443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.930723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.760792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.606667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.732389] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5161.743365] Lustre: Skipped 33 previous similar messages [ 5161.764492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5161.768408] Lustre: Skipped 33 previous similar messages [ 5161.770811] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5161.772875] Lustre: Skipped 33 previous similar messages [ 5161.775198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.782678] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5161.792113] Lustre: Skipped 33 previous similar messages [ 5162.648220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.635425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.752538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.574890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.634977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.563379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.612166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.559981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.671533] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5170.693505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.704800] LustreError: 172264:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5172.001117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.113333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.091527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.125590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.047199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.043394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.101718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.120733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.104983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.357313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.352389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.309465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.353828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.271590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.220697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.404137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.400333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.345769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.568481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.896467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.189255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.148351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.177684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.087292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.961946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.717969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.733322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.629932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.577363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.367765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.269826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.081455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.125555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.204880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.115059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.064397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.003921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.947600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.980123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.082356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.145844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.038519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.019603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.026031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.873349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.831458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.591571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.523764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.349119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.291587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.427543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.324603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.257173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.095441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.084858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.036814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5226.042722] Lustre: Skipped 64 previous similar messages [ 5226.054695] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5226.070757] Lustre: Skipped 64 previous similar messages [ 5226.073722] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5226.077528] Lustre: Skipped 64 previous similar messages [ 5226.080806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.089884] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5226.101258] Lustre: Skipped 64 previous similar messages [ 5227.021322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.836089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.733951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.585411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.553110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.385338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.300503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.085567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.181572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.360905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.699445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.815883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.727494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.742689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.876514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.880856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.981679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.072749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.107809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.186052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.164196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.168375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.207681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.177699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.257734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.215610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.194223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.400406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.376745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.299889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.021379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.008692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.092113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.391641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.311187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.328686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.374819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.323412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.435337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.461455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.458545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.562495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.515839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.385178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.242830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.279479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.129234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.111391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.969995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.871825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.693505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.738413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.540760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.450466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.492709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.576263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.638280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.781919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.164410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.388924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.734815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.803058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.924077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.075926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.138954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.046482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.916972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.966840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.766224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.718186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.587282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.355909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.069810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.913659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.752376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.620532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.467284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.233635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.185819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.266769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.188966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.180846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.291989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.472087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.641297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.123261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.594714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.599371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.860677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.586953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.730570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.905815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.342491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.390444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.064875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.652373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.906320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.884650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.142502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.107038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.399271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.866952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.923976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.844273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.856674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.898106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.707387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.607714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.663226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.540395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.569231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.501567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.629593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.457692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.640578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.752595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.743966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.614796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.560360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.482315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.437078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.531210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.266029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.207443] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5354.211394] Lustre: Skipped 123 previous similar messages [ 5354.236923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5354.249352] Lustre: Skipped 123 previous similar messages [ 5354.254850] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5354.260799] Lustre: Skipped 123 previous similar messages [ 5354.265068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.277613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5354.280443] Lustre: Skipped 123 previous similar messages [ 5355.307179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.343426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.311093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.437389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.327908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.316294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.175260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.070389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.028970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.931778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.081810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.053626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.149789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.191439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.349696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.124919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.011096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.981279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.778224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.484722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.230716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.045593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.091162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.880617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.854566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.879095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.999719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.140265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.094608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.178111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.271947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.380321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.288654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.110338] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5387.119244] LustreError: 172264:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5387.129173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.911936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.814913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.579530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.606639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.414655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.237301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.392382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.425740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.316643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.378673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.321655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.227618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.345214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.181682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.070469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.771260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.678595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.524638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.467025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.215721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.091642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.061231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.933450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.773210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.647922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.452709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.745563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.341579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.793533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.002201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.229780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.547519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.925030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.890708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.015518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.185906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.083291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.742806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.448538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.211897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.032705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.011198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.029760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.533124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.880516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.774166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.393597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.394655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.970550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.406958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.641700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.534854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.999848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.083471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.012549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.847907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.918563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.918122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.934270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.709625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.483144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.433203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.645568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.638413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.588789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.717523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.934165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.086571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.088700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.394801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.595217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.553467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.769600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.774822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.069615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.507864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.923329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.009204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.172506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.337706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.380841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.279764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.491737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.878982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.854257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.886194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.678512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.733511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.978337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.144680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.189531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.104500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.198255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.018847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.961942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.806794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.817730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.063901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.259541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.680111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.763383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.730118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.671338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.605750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.105883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.866482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.811607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.618956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.529319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.529313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.517585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.352478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.035793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.653892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.476420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.456463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.285965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.942943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.665970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.402125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.305782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.230552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.129518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.867278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.916525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.765666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.751300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.646598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.606810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.492118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.534650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.671199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.917418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.922257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.920889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.812687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.986244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.830498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.776629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.802316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.809222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.784154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.731912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.513853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.491751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.307797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.190975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.092098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.210943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.384719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.646576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.460378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.296531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.169912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.945040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.922378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.782541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.729773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.883919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.044964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.082649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.049828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.955488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.966233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.789566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.822546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.796562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.642924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.582904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.546557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.546040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.366154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.280759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.119677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.416925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.593972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.812928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.977449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.051932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.037841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.285163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.140046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.187206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.974199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.809459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.655216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.488490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.321835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.267737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.188674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.054483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.888719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.588615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.455919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.240774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.082457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.797706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.726340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.485802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.709217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.512554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.840543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.936315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.033411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.988427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.005253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.051864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.966652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.666080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.473137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.341720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.148751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.003132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.022884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.854482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5610.859382] Lustre: Skipped 248 previous similar messages [ 5610.881527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5610.892274] Lustre: Skipped 248 previous similar messages [ 5610.897701] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5610.902912] Lustre: Skipped 248 previous similar messages [ 5610.905685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.913487] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5610.925666] Lustre: Skipped 248 previous similar messages [ 5611.817385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.673642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.468908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.293534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.366368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.188023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.437064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.716786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.793134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.813155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.569411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.478815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.614587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.534829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.374482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.336154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.204683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.065523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.185987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.389374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.937105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.539707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.715724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.760952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.791115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.001910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.934744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.048279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.223141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.325471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.560920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.715185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.042775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.043197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.207811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.302902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.232074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.445902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.415297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.461497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.325367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.474269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.361654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.502049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.612875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.763708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.850874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.808306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.840461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.855697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.964809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.891681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.801955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.744751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.033513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.954990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.102359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.997746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.017187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.018710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.873851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.923886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.029637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.252431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.527571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.582216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.607389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.559804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.571784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.603399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.670427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.974961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.081067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.157061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.415854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.702384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.896312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.914282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.044369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.145215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.049878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.922711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.068976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.360562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.212441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.320970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.343477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.306457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.283862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.241305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.424386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.561987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.779802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.796112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.059892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.260309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.158467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.161691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.124985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.148435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.295070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.604487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.715364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.874479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.833958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.722406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.704864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.832052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.067946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.027341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.270034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.348690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.427385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.579806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.674526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.989618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.102925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.067280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.209479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.148351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.166446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.444840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.453686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.610750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.680982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.735118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.719145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.926071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.800669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.857854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.966465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.918912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.037110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.965644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.037522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.940480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.006883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.918732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.083395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.966548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.015256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.236241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.382800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.500919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.608449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.711965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.959273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.893333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.875830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.615151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.496238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.525412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.404866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.550706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.503373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.482517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.523451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.556113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.066254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.646933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.910920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.244670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.456238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.536539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.442812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.526280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.610474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.317424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.283777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.187378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.041722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.849487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.690591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.594154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.593394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.622243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.535198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.657135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.652451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.858650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.923900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.888818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.880153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.731251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.670742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.791744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.903746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.688775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.718534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.579845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.422587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.234095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.017491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.885769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.825082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.662370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.809053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.817121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.012701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.932283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.793809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.632641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.914428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.183496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.183618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.674538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.954628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.265715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.326874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.567249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.601594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.755154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.726711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.882194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.989472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.046601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.998096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.081472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.195374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.110232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.122500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.143541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.340109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.166652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.038243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.081400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.279631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.581398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.526261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.259050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.902796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.661718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.630894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.870866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.726184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.478404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.341515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.953539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.456626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.807489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.415594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.277851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.309583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.284726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.344466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.326040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.216829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.057712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.086788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.050054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.144347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.161785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.185479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.146543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.293674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.026204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.889860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.865798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.699132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.609828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.827045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.861876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.950540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.926426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.029708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.842671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.704310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.562843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.687138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.679482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.628328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.544521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.419053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.244429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.189923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.232102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.101659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.963565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.978379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.986424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.177390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.101626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.346468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.385506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.739598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.223426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.465714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.216415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.143756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.457889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.331825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.616990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.778913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.221200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.407330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.769808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.301172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.522430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.598838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.553546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.786300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.885388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.084804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.291354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.341772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.335232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.623281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.665086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.249518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.573220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.874894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.115746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.972167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.260694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.642907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.717108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.080315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.470791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.883312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.280292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.606893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.672351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.658172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.681554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.733797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.909595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.625854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.674352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.262065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.238092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.432886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.553562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.762057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.960679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.787469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.491421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.432500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.338798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.188941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.177589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.240095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.116859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.971788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.961425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.797985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.728743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.916610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.872067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.961595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.173711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.209433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.212971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.912478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.543952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.324482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.096370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.052132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.913435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.936407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.810213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.684291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.564844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.303665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.211725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.136247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.055443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.971301] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6006.978869] LustreError: 172264:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6006.994823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.945648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.849905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.685473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.587270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.377806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.293987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.171332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.080501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.000393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.011987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.819612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.668677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.471681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.200161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.047183] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6020.057818] LustreError: 172264:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6020.072749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.973466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.814157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.635945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.500412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.201828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.241484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.085166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.939713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.728941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.837811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.649579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.527103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.760219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.768045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.706837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.437449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.294527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.229918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.092140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.971689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.812995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.640568] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6039.654877] LustreError: 172264:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6039.660805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.417663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.161601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.016954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.027451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.851299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.709637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.540720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.359103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.125504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.008486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.002865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.055814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.001305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.988864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.783414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.678045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.540830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.322837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.199241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.298322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.171117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.166046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.142420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.138446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.506407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.825856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.096940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.260780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.387544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.210264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.344477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.665775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.290981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.360238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.348504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.659627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.545164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.203122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.260255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.606532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.857852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.006231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.263153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.625168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.036981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.430876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.183620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.566291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.459830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.946727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.131848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.413925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.389226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.355870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.495977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.468134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.454849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.477911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.371055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.196781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.134295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.270392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.863126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.645952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.495339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.546954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.872138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.084058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.356895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.559781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.692427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.848900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.848229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.989710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.843083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.786238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.568834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.569531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.329775] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6123.334515] Lustre: Skipped 482 previous similar messages [ 6123.351478] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6123.362698] Lustre: Skipped 482 previous similar messages [ 6123.366794] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6123.370059] Lustre: Skipped 482 previous similar messages [ 6123.374572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.387765] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6123.399096] Lustre: Skipped 482 previous similar messages [ 6124.207786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.128059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.183135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.364744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.324594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.511416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.461764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.606632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.870697] LustreError: 174541:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6463.877748] LustreError: 172264:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6874.091890] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6874.111552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6874.120079] Lustre: Skipped 8 previous similar messages [ 6874.138088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6874.596790] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6874.610653] LustreError: Skipped 1 previous similar message [ 6877.743642] Lustre: server umount lustre-OST0000 complete [ 6884.836065] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6884.837654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6884.851489] Lustre: Skipped 2 previous similar messages [ 6888.418876] Lustre: server umount lustre-MDT0000 complete [ 6889.958621] LustreError: 172245:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6889.976809] LustreError: 172245:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6892.193776] LustreError: 172225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790952725 with bad export cookie 12842730501961384064 [ 6892.203371] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6892.206252] LustreError: 172225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6892.732733] Lustre: server umount lustre-MDT0001 complete [ 6906.507553] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 6909.327429] Key type lgssc unregistered [ 6909.600484] LNet: 227460:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6909.605683] LNetError: 227460:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6909.622276] LNet: Removed LNI 192.168.201.139@tcp [ 6910.438496] Key type .llcrypt unregistered [ 6910.443868] Key type ._llcrypt unregistered [ 6935.367838] Key type ._llcrypt registered [ 6935.370593] Key type .llcrypt registered [ 6935.458571] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 6936.846686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6937.036407] alg: No test for adler32 (adler32-zlib) [ 6938.217943] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 6938.465248] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 6940.159246] Key type lgssc registered [ 6941.863322] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6952.371953] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 6952.396190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6954.030223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6960.179500] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 6963.195475] LustreError: 228891:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6963.224663] LustreError: 228891:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6968.295572] LustreError: 228892:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6969.143208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6969.668942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6974.918695] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 6984.092670] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6991.144163] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6999.788863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7000.241031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7008.932602] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7016.744623] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7023.796911] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7031.700337] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7033.713748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7033.725567] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7033.731914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.755244] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7033.755498] LustreError: 228911:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7037.809793] Lustre: lustre-OST0000: Not available for connect from 192.168.201.39@tcp (stopping) [ 7039.456982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7039.476664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7039.488343] Lustre: Skipped 1 previous similar message [ 7039.502895] Lustre: Skipped 1 previous similar message [ 7041.910604] Lustre: server umount lustre-OST0000 complete [ 7053.791198] LustreError: 228911:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7076.912147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7077.457697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7079.464508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.478294] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.499133] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7084.611330] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7094.235212] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7096.070806] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7102.569734] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8aa188496800.ost_server_uuid 50 [ 7104.309928] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8aa188496800.ost_server_uuid in IDLE state after 0 sec [ 7109.613094] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7109.618506] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7109.625415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7110.631840] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7113.189480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7113.203754] Lustre: Skipped 2 previous similar messages [ 7115.927703] Lustre: server umount lustre-OST0000 complete [ 7123.426886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7123.431495] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7123.438081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7123.466511] Lustre: Skipped 2 previous similar messages [ 7126.458418] Lustre: server umount lustre-MDT0000 complete [ 7128.546021] LustreError: 228887:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7128.586636] LustreError: 228887:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7130.145160] LustreError: 228872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790952963 with bad export cookie 2348269525517280889 [ 7130.153344] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7130.164989] LustreError: 228872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7130.605729] Lustre: server umount lustre-MDT0001 complete [ 7143.433725] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 7146.553223] Key type lgssc unregistered [ 7147.079456] LNet: 234125:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7147.099821] LNetError: 234125:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7147.127601] LNet: Removed LNI 192.168.201.139@tcp [ 7148.652446] Key type .llcrypt unregistered [ 7148.654084] Key type ._llcrypt unregistered [ 7186.798701] Key type ._llcrypt registered [ 7186.801436] Key type .llcrypt registered [ 7186.989460] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 7188.417087] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7188.523502] alg: No test for adler32 (adler32-zlib) [ 7189.886580] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 7190.293271] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 7192.047486] Key type lgssc registered [ 7193.969443] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7203.477328] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7203.528853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7204.891342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7209.487235] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7215.080226] LustreError: 235870:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7215.111319] LustreError: 235870:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7217.897728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7218.174876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7223.274379] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7226.871189] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7233.366741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7233.833605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7239.901294] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7248.537937] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7248.687553] Lustre: lustre-OST0001: new disk, initializing [ 7248.690977] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7248.696114] Lustre: 238175:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7248.812337] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7249.867549] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7255.080855] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7258.753097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7258.769292] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7258.844174] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7266.378312] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7273.080927] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7294.949131] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7294.972808] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7294.979978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7295.004646] Lustre: Skipped 1 previous similar message [ 7295.024486] Lustre: Skipped 3 previous similar messages [ 7298.222195] Lustre: server umount lustre-MDT0000 complete [ 7300.072783] LustreError: 235870:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7300.089958] LustreError: 235870:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7302.139125] LustreError: 235852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790953135 with bad export cookie 6502278668905488989 [ 7302.150809] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7302.878276] Lustre: server umount lustre-MDT0001 complete [ 7316.524297] Lustre: server umount lustre-OST0000 complete [ 7330.878658] Lustre: server umount lustre-OST0001 complete [ 7352.350579] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 7363.033310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7363.566820] LustreError: 241292:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7363.680370] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7367.965474] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7368.689222] LustreError: 241293:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7373.793857] LustreError: 241292:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7375.740059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7376.231221] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7381.805926] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7385.419553] Lustre: 242434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7393.253609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7393.553564] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7399.842802] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7401.766453] LustreError: 242789:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7408.893778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7410.211154] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7415.680209] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7422.650562] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7425.693164] Lustre: 244306:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7437.793579] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7437.803251] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7437.825297] Lustre: Skipped 2 previous similar messages [ 7437.839637] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7440.865402] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7440.866648] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7440.900148] Lustre: Skipped 2 previous similar messages [ 7443.042452] Lustre: server umount mylustre-MDT0000 complete [ 7445.997855] LustreError: 242026:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7446.025349] LustreError: 242026:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 7447.148484] LustreError: 241269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790953280 with bad export cookie 6502278668905491761 [ 7447.158246] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7447.160439] LustreError: 241269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 7447.492187] Lustre: server umount mylustre-MDT0001 complete [ 7461.879739] Lustre: server umount mylustre-OST0000 complete [ 7476.082854] Lustre: server umount mylustre-OST0001 complete [ 7498.288283] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 7507.910911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7508.310655] LustreError: 246625:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7508.414810] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7508.417700] Lustre: Skipped 1 previous similar message [ 7512.740458] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7520.201881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7524.797384] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7527.231181] Lustre: 247766:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7532.999802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7534.538940] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7539.513107] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7548.743468] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7558.521561] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7567.858675] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7571.005941] Lustre: 249637:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7580.140681] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7580.162743] Lustre: Skipped 2 previous similar messages [ 7580.163047] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7580.189664] Lustre: Skipped 6 previous similar messages [ 7584.312987] Lustre: server umount tfs-MDT0000 complete [ 7585.258606] LustreError: 246626:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7585.271880] LustreError: 246626:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 10 previous similar messages [ 7588.480580] LustreError: 246600:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790953421 with bad export cookie 6502278668905493427 [ 7588.485281] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7588.492922] LustreError: 246600:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7588.974456] Lustre: server umount tfs-MDT0001 complete [ 7602.388315] Lustre: server umount tfs-OST0000 complete [ 7615.659524] Lustre: server umount tfs-OST0001 complete [ 7637.761297] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 7648.429888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7648.990143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7648.998399] Lustre: Skipped 3 previous similar messages [ 7653.795249] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7663.379995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7663.584359] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7668.773306] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7671.878504] Lustre: 253029:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7679.115990] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7689.197247] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7694.068187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7698.526294] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7705.201462] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7714.110891] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7718.476175] Lustre: 254899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7727.900419] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:05:57 (1790953557) [ 7750.117893] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7750.144383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7750.160034] Lustre: Skipped 1 previous similar message [ 7761.376055] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7761.784343] Lustre: server umount lustre-MDT0000 complete [ 7765.496879] LustreError: 251884:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7765.535532] LustreError: 251884:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [ 7766.756729] LustreError: 251865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790953599 with bad export cookie 6502278668905494869 [ 7766.764603] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7766.780095] LustreError: 251865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7770.613012] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7770.639403] Lustre: Skipped 3 previous similar messages [ 7770.650400] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7770.653090] Lustre: Skipped 9 previous similar messages [ 7781.343265] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7781.631408] Lustre: server umount lustre-MDT0001 complete [ 7793.867088] Lustre: server umount lustre-OST0000 complete [ 7804.696454] Lustre: server umount lustre-OST0001 complete [ 7812.287737] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 7822.357974] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 7858.446608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7858.649305] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7858.671653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7858.736359] Lustre: lustre-MDT0000: new disk, initializing [ 7858.785924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7858.788636] Lustre: Skipped 3 previous similar messages [ 7858.802941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7862.975636] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7874.521874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7874.665604] Lustre: 258093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7874.714045] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7874.717256] Lustre: Skipped 1 previous similar message [ 7874.800847] Lustre: lustre-MDT0001: new disk, initializing [ 7874.953264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7874.970034] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7879.856258] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7889.846603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7890.152900] Lustre: lustre-OST0000: new disk, initializing [ 7890.155908] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7890.162911] Lustre: 259674:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7891.445239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7891.464413] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7891.632333] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7897.358261] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7907.785299] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7907.946946] Lustre: lustre-OST0001: new disk, initializing [ 7907.950931] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7907.957948] Lustre: 260544:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7909.187178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7909.207477] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7909.297698] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7913.959519] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 7926.066726] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:09:17 (1790953757) [ 7942.328809] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:09:33 (1790953773) [ 7957.992688] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7958.009922] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7958.015670] Lustre: Skipped 1 previous similar message [ 7958.020763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7958.023024] Lustre: Skipped 4 previous similar messages [ 7963.591917] Lustre: server umount lustre-OST0000 complete [ 7974.663703] Lustre: server umount lustre-MDT0000 complete [ 7979.373942] LustreError: 258897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790953812 with bad export cookie 6502278668905496199 [ 7979.387792] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7979.410645] LustreError: 258897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7986.205928] Lustre: server umount lustre-MDT0001 complete [ 7998.018215] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 8072.114729] Lustre: server umount lustre-OST0001 complete [ 8078.866456] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 8095.542119] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8131.641448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8131.836466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8131.859569] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8131.916976] Lustre: lustre-MDT0000: new disk, initializing [ 8131.974728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8131.980701] Lustre: Skipped 3 previous similar messages [ 8131.992205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8136.564690] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8146.150447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8146.320679] Lustre: 265396:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8146.550364] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8150.390478] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8159.753698] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8165.767921] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8173.276303] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8173.582407] Lustre: lustre-OST0000: new disk, initializing [ 8173.586712] Lustre: Skipped 1 previous similar message [ 8173.592337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8173.595971] Lustre: Skipped 2 previous similar messages [ 8173.605532] Lustre: 267367:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8175.293431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8175.301235] Lustre: Skipped 1 previous similar message [ 8175.309908] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8175.458477] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8180.901543] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8190.538702] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8201.193215] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8201.194507] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8201.215642] Lustre: Skipped 6 previous similar messages [ 8201.215902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8201.215906] Lustre: Skipped 8 previous similar messages [ 8201.237944] LustreError: Skipped 1 previous similar message [ 8203.537071] Lustre: server umount lustre-OST0000 complete [ 8207.842203] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8222.175286] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8222.552485] Lustre: server umount lustre-MDT0000 complete [ 8226.786288] LustreError: 265410:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8226.815470] LustreError: 265410:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 16 previous similar messages [ 8226.940488] LustreError: 265388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790954059 with bad export cookie 6502278668905497669 [ 8226.941371] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8226.972032] LustreError: 265388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8227.279274] Lustre: server umount lustre-MDT0001 complete [ 8240.725691] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 8244.030819] Key type lgssc unregistered [ 8244.364950] LNet: 269127:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8244.373509] LNetError: 269127:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8244.403483] LNet: Removed LNI 192.168.201.139@tcp [ 8245.446379] Key type .llcrypt unregistered [ 8245.448586] Key type ._llcrypt unregistered [ 8275.351758] Key type ._llcrypt registered [ 8275.358565] Key type .llcrypt registered [ 8275.482605] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8276.850234] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8276.882268] alg: No test for adler32 (adler32-zlib) [ 8278.034686] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 8278.308473] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 8279.967352] Key type lgssc registered [ 8281.196071] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8288.902511] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8288.915870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8290.270148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8295.763907] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8300.515068] LustreError: 270604:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8300.527632] LustreError: 270604:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8304.204513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8304.643813] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8309.312406] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8317.108942] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8323.497954] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8331.507440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8332.049766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8335.171944] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8338.796537] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8347.413858] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8348.325447] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8350.695030] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8350.698877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8350.735719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8351.778966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8356.621554] Lustre: server umount lustre-OST0000 complete [ 8360.931435] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8360.948099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8360.967901] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8360.978575] Lustre: Skipped 1 previous similar message [ 8361.954148] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8366.389851] Lustre: server umount lustre-MDT0000 complete [ 8367.077558] LustreError: 270604:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8367.097977] LustreError: 270604:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8370.560520] LustreError: 273132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790954203 with bad export cookie 5855403167816733390 [ 8370.566554] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8370.573354] LustreError: 273132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8370.978788] Lustre: server umount lustre-MDT0001 complete [ 8385.004381] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 8388.287568] Key type lgssc unregistered [ 8388.557809] LNet: 274186:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8388.570326] LNetError: 274186:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8388.607850] LNet: Removed LNI 192.168.201.139@tcp [ 8389.439889] Key type .llcrypt unregistered [ 8389.442029] Key type ._llcrypt unregistered [ 8408.155777] Key type ._llcrypt registered [ 8408.157498] Key type .llcrypt registered [ 8408.268553] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8409.861717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8409.906268] alg: No test for adler32 (adler32-zlib) [ 8410.947257] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 8411.170189] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 8412.895316] Key type lgssc registered [ 8413.919564] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8422.338268] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8422.357183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8437.922449] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8443.039445] LustreError: 275395:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.139@tcp: failed processing log, type 4: rc = -110 [ 8469.663613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8476.148680] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8483.704554] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8486.097610] Lustre: server umount lustre-OST0000 complete [ 8504.804977] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 8508.465509] Key type lgssc unregistered [ 8508.772464] LNet: 277030:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8508.781206] LNetError: 277030:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8508.800444] LNet: Removed LNI 192.168.201.139@tcp [ 8509.655684] Key type .llcrypt unregistered [ 8509.659197] Key type ._llcrypt unregistered [ 8547.467391] Key type ._llcrypt registered [ 8547.471736] Key type .llcrypt registered [ 8547.597882] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8548.529335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8548.557587] alg: No test for adler32 (adler32-zlib) [ 8549.780659] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 8549.968360] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 8551.680140] Key type lgssc registered [ 8553.039966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8565.459697] loop0: detected capacity change from 0 to 204800000 [ 8565.670270] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8565.682500] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8566.032367] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8567.018245] loop0: detected capacity change from 0 to 204800000 [ 8567.144570] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8567.156435] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8567.518193] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8568.538430] loop0: detected capacity change from 0 to 204800000 [ 8568.795382] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8568.816912] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8568.960220] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8570.126982] loop0: detected capacity change from 0 to 204800000 [ 8570.332627] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8571.410216] loop0: detected capacity change from 0 to 204800000 [ 8571.451927] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8575.803429] loop0: detected capacity change from 0 to 204800000 [ 8575.837032] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8579.762694] loop0: detected capacity change from 0 to 204800000 [ 8579.798021] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8583.749139] loop0: detected capacity change from 0 to 204800000 [ 8583.795295] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8588.554270] loop0: detected capacity change from 0 to 204800000 [ 8588.623203] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8588.687839] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8588.799211] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8588.806330] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8588.822504] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8588.843842] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8588.857945] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8588.866808] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8588.876949] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8588.887263] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8588.900087] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8588.911242] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8588.926645] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8588.937214] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8588.948953] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8588.960573] Lustre: 280471:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8594.935868] Lustre: server umount MGS complete [ 8595.878547] loop0: detected capacity change from 0 to 204800000 [ 8595.984178] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8596.514046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8596.567675] Lustre: lustre-MDT0000: Aborting client recovery [ 8597.803851] loop1: detected capacity change from 0 to 204800000 [ 8598.056958] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8598.141145] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8598.159589] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8598.202121] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8598.237830] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8598.255354] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8598.266863] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8598.272073] Lustre: 280902:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8598.285704] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8598.513797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8598.550922] Lustre: lustre-MDT0001: Aborting client recovery [ 8599.530929] loop2: detected capacity change from 0 to 204800000 [ 8599.811126] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8599.935668] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8599.948297] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8599.974174] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8599.983954] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8599.998976] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8600.007068] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8600.015685] Lustre: 280983:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8600.129985] Lustre: Found index 0 for lustre-OST0000, updating log [ 8600.168918] Lustre: 281001:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8600.227816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8600.245609] Lustre: lustre-OST0000: Aborting recovery [ 8601.726575] loop3: detected capacity change from 0 to 204800000 [ 8601.905201] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8602.053645] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8602.072781] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8602.098433] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8602.128345] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8602.139959] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8602.146729] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8602.156310] Lustre: 281070:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8602.180823] Lustre: Found index 1 for lustre-OST0001, updating log [ 8602.205919] Lustre: 281076:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8602.314845] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8602.345795] Lustre: lustre-OST0001: Aborting recovery [ 8606.461949] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8606.484076] LustreError: 280846:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8606.504029] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8606.550566] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8606.600941] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8608.502060] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8608.504368] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8611.037132] LustreError: 280813:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3270us: rc = -2 [ 8611.550719] LustreError: 280813:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 564us: rc = -2 [ 8611.579628] LustreError: 280813:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8629.225613] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8629.226278] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8629.228382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8629.258041] Lustre: Skipped 3 previous similar messages [ 8631.515336] Lustre: server umount lustre-MDT0000 complete [ 8632.374046] LustreError: 280796:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790954465 with bad export cookie 4914949875397523632 [ 8632.388642] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8634.337283] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8634.342445] LustreError: 281283:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8634.369240] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8634.369248] Lustre: Skipped 3 previous similar messages [ 8634.418740] LustreError: 281283:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8637.345760] LustreError: 280811:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8637.345805] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8637.406280] Lustre: Skipped 2 previous similar messages [ 8639.596256] Lustre: server umount lustre-MDT0001 complete [ 8655.329745] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8655.507554] Lustre: server umount lustre-OST0000 complete [ 8662.892229] Lustre: server umount lustre-OST0001 complete [ 8671.852807] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 11:21:42 (1790954502) [ 8694.665363] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8737.203141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8737.613181] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8737.671479] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8737.779080] Lustre: lustre-MDT0000: new disk, initializing [ 8737.918077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8737.975454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8744.787399] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8753.235784] Lustre: server umount lustre-MDT0000 complete [ 8772.921882] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:23:23 (1790954603) [ 8804.765545] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 8819.071859] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 8864.850950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8865.284720] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8865.353897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8865.360935] Lustre: Skipped 1 previous similar message [ 8865.462311] Lustre: lustre-MDT0000: new disk, initializing [ 8865.590519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8865.611369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8871.700343] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8884.552677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8884.662152] Lustre: 288185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8884.699728] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8884.703307] Lustre: Skipped 1 previous similar message [ 8884.751422] Lustre: lustre-MDT0001: new disk, initializing [ 8884.814024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8884.846823] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8884.852455] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8888.613178] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8898.441738] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8904.503609] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8912.371599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8912.813542] Lustre: lustre-OST0000: new disk, initializing [ 8912.817122] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8912.823039] Lustre: 290153:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8918.648802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8918.657898] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8918.804244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8920.260357] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 8930.038305] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8935.761866] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8947.258954] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8952.871253] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9006.059510] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9006.062398] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9006.065197] Lustre: Skipped 1 previous similar message [ 9006.068779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9006.073133] LustreError: Skipped 1 previous similar message [ 9011.169568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9011.175897] Lustre: Skipped 1 previous similar message [ 9011.799884] Lustre: server umount lustre-OST0000 complete [ 9016.301195] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9016.332390] Lustre: Skipped 2 previous similar messages [ 9021.410331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9021.412553] Lustre: Skipped 4 previous similar messages [ 9021.940915] Lustre: server umount lustre-MDT0000 complete [ 9025.940728] LustreError: 288177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790954858 with bad export cookie 4914949875397526698 [ 9025.946319] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9025.973344] LustreError: 288177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9026.617453] Lustre: server umount lustre-MDT0001 complete [ 9035.566851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9040.450343] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9046.428442] Lustre: server umount MGS complete [ 9058.283403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9059.020767] LustreError: 294018:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9059.059257] LustreError: 294018:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9059.062674] LustreError: 294031:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9059.203916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9059.222379] Lustre: Skipped 1 previous similar message [ 9063.627192] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9064.422223] LustreError: 294019:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9068.518287] LustreError: 294018:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9073.349435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9079.099812] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9088.523974] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9094.059834] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9100.027800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9109.083910] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9122.487546] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9155.044177] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9155.046672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9155.051293] LustreError: Skipped 1 previous similar message [ 9155.058808] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9155.064185] Lustre: Skipped 1 previous similar message [ 9155.081075] Lustre: Skipped 1 previous similar message [ 9167.328354] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9167.659461] Lustre: server umount lustre-OST0000 complete [ 9175.521327] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9175.572060] Lustre: Skipped 1 previous similar message [ 9187.296445] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9188.345251] Lustre: server umount lustre-MDT0000 complete [ 9190.889690] LustreError: 294019:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9190.902851] LustreError: 294019:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9193.507220] LustreError: 293999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790955026 with bad export cookie 4914949875397528525 [ 9193.515333] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9193.521774] LustreError: 293999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9193.897430] Lustre: server umount lustre-MDT0001 complete [ 9205.447592] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 9208.551266] Key type lgssc unregistered [ 9208.847790] LNet: 297968:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9208.868993] LNetError: 297968:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9208.895990] LNet: Removed LNI 192.168.201.139@tcp [ 9210.048328] Key type .llcrypt unregistered [ 9210.053948] Key type ._llcrypt unregistered [ 9250.941276] Key type ._llcrypt registered [ 9250.943088] Key type .llcrypt registered [ 9251.107912] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 9268.923960] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 9270.359096] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9270.706463] alg: No test for adler32 (adler32-zlib) [ 9272.494865] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 9273.097272] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 9274.983291] Key type lgssc registered [ 9276.696525] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9322.855169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9322.892265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9324.267489] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9324.323444] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9324.418249] Lustre: lustre-MDT0000: new disk, initializing [ 9324.515611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9324.542734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9329.526440] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9341.615608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9341.809575] Lustre: 301947:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9341.900306] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9341.905254] Lustre: Skipped 1 previous similar message [ 9341.972368] Lustre: lustre-MDT0001: new disk, initializing [ 9342.128483] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9342.179518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9342.195602] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9347.527211] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9358.151854] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9363.764763] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9370.826715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9371.232699] Lustre: lustre-OST0000: new disk, initializing [ 9371.236656] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9371.242439] Lustre: 303913:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9371.322485] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9378.563991] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9380.529436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9380.552228] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9380.785604] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9390.461986] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9398.412412] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9410.463117] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9414.595177] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9419.831545] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9462.753026] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9462.761119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9462.763875] LustreError: Skipped 1 previous similar message [ 9462.764861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9462.792170] Lustre: Skipped 1 previous similar message [ 9464.548411] Lustre: server umount lustre-OST0000 complete [ 9470.434424] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9470.450111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9470.482847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9470.492828] Lustre: Skipped 1 previous similar message [ 9473.007204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9473.016200] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9475.383473] Lustre: server umount lustre-MDT0000 complete [ 9478.802830] LustreError: 301939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790955311 with bad export cookie 9785577732231269854 [ 9478.807239] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9478.823699] LustreError: 301939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9479.414655] Lustre: server umount lustre-MDT0001 complete [ 9490.592064] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9496.552171] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9501.841829] Lustre: server umount MGS complete [ 9514.597587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9515.023664] LustreError: 307681:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9515.047878] LustreError: 307681:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9515.068897] LustreError: 307694:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9515.134170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9519.582798] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9520.614803] LustreError: 307682:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9524.713708] LustreError: 307681:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9529.531682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9529.895272] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9536.692092] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9548.562355] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9560.639276] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9569.679389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9570.137678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9576.780297] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9586.448395] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9615.329495] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9615.346424] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9615.362586] Lustre: Skipped 1 previous similar message [ 9615.362889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9615.362892] Lustre: Skipped 1 previous similar message [ 9620.454497] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9620.466216] Lustre: Skipped 2 previous similar messages [ 9629.151160] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9629.509616] Lustre: server umount lustre-OST0000 complete [ 9635.822079] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9635.826880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9635.842870] Lustre: Skipped 1 previous similar message [ 9635.868170] Lustre: Skipped 3 previous similar messages [ 9640.545994] Lustre: server umount lustre-MDT0000 complete [ 9640.938728] LustreError: 307677:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9640.971890] LustreError: 307677:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9645.676857] LustreError: 310412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790955478 with bad export cookie 9785577732231271681 [ 9645.688550] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9645.704116] LustreError: 310412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9646.392770] Lustre: server umount lustre-MDT0001 complete [ 9660.539353] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [ 9664.229419] Key type lgssc unregistered [ 9664.638604] LNet: 311632:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9664.644385] LNetError: 311632:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9665.706483] LNet: Removed LNI 192.168.201.139@tcp [ 9666.884051] Key type .llcrypt unregistered [ 9666.888261] Key type ._llcrypt unregistered [ 9691.464717] Key type ._llcrypt registered [ 9691.466437] Key type .llcrypt registered [ 9691.603561] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [ 9708.109179] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 9709.339378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9709.569319] alg: No test for adler32 (adler32-zlib) [ 9710.777107] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [ 9711.074319] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [ 9712.791209] Key type lgssc registered [ 9714.148846] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9742.068598] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [ 9756.933629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9756.988644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9758.261857] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9758.305715] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9758.439486] Lustre: lustre-MDT0000: new disk, initializing [ 9758.552479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9758.574970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9763.744324] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9777.579417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9778.014285] Lustre: lustre-OST0000: new disk, initializing [ 9778.018536] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9778.026487] Lustre: Skipped 1 previous similar message [ 9778.029708] Lustre: 315698:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9778.123921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9786.447496] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [ 9788.576292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9788.596322] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9788.628476] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10608.099834] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10608.111864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10613.310781] Lustre: server umount lustre-MDT0000 complete [10616.959259] LustreError: 314885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790956449 with bad export cookie 12041939279332751497 [10616.973263] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10617.098850] Lustre: server umount lustre-OST0000 complete [10642.432538] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [10652.105799] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [10694.578277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10694.752400] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10694.773422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10694.834140] Lustre: lustre-MDT0000: new disk, initializing [10694.890109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10694.907983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10701.593094] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10714.285220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10714.469778] Lustre: 320031:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10714.543684] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10714.547621] Lustre: Skipped 1 previous similar message [10714.703875] Lustre: lustre-MDT0001: new disk, initializing [10714.864694] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10714.937043] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10714.957683] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10720.356830] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10733.273903] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10740.841152] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10749.591245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10749.908723] Lustre: lustre-OST0000: new disk, initializing [10749.911988] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10749.921242] Lustre: 322001:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10750.013645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10757.159459] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10757.763776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10757.776811] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10757.974622] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10766.780369] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10771.911172] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10772.141591] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10776.022429] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10776.225161] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10778.592309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10778.598381] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10778.599293] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10778.607212] Lustre: Skipped 1 previous similar message [10778.616561] Lustre: Skipped 1 previous similar message [10783.716830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10783.731679] Lustre: Skipped 1 previous similar message [10784.471605] Lustre: server umount lustre-OST0000 complete [10791.905519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10791.919638] LustreError: Skipped 1 previous similar message [10791.926008] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10791.937655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10793.960950] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10795.296336] Lustre: server umount lustre-MDT0000 complete [10799.088931] LustreError: 320042:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10799.105939] LustreError: 320042:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10800.584796] LustreError: 320022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790956633 with bad export cookie 12041939279335692309 [10800.610867] LustreError: 320022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10800.628087] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10801.072786] Lustre: server umount lustre-MDT0001 complete [10805.606053] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10808.065269] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:57:18 (1790956638) [10816.084901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10816.920430] LustreError: 324314:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10817.062216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10822.119075] LustreError: 324315:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10822.141707] LustreError: 324315:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10822.219122] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10826.211151] LustreError: 324314:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10831.335798] LustreError: 324315:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10832.947888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10833.477498] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10839.261515] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10847.543884] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10853.816664] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10861.169536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10861.483967] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10868.691782] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10876.263976] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10884.110491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10884.298185] Lustre: lustre-OST0001: new disk, initializing [10884.301940] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10884.314680] Lustre: 327150:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10884.408596] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10892.447097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10892.454211] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10892.483677] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [10892.524918] LustreError: 327499:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10893.602921] LustreError: 327499:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10893.614944] LustreError: 327499:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10894.692115] LustreError: 327512:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10896.742538] LustreError: 327512:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10896.761387] LustreError: 327512:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10900.906923] LustreError: 327512:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10900.927673] LustreError: 327512:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10902.771799] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10902.996419] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10909.011573] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8aa192f4f000.ost_server_uuid 50 [10909.166932] LustreError: 327499:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10909.175916] LustreError: 327499:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10910.804392] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8aa192f4f000.ost_server_uuid in FULL state after 0 sec [10915.920806] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8aa192f4f000.ost_server_uuid 50 [10917.406446] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8aa192f4f000.ost_server_uuid in FULL state after 0 sec [10919.535274] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10933.648502] Lustre: lustre-OST0001: Not available for connect from 192.168.201.39@tcp (stopping) [10933.657034] Lustre: Skipped 2 previous similar messages [10933.755387] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10938.283942] Lustre: server umount lustre-OST0001 complete [10938.732233] LustreError: 326214:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10945.504771] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10945.520113] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10945.538274] Lustre: Skipped 1 previous similar message [10945.543171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10945.550127] Lustre: Skipped 2 previous similar messages [10951.342648] Lustre: server umount lustre-OST0000 complete [10962.814990] Lustre: server umount lustre-MDT0000 complete [10964.449468] LustreError: 324315:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10964.466734] LustreError: 324315:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [10967.364066] LustreError: 324294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790956800 with bad export cookie 12041939279335693198 [10967.366860] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10967.377095] LustreError: 324294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10967.691073] Lustre: server umount lustre-MDT0001 complete [10978.640163] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [10981.427308] Key type lgssc unregistered [10981.783399] LNet: 329841:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10981.792897] LNetError: 329841:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10981.804532] LNet: Removed LNI 192.168.201.139@tcp [10982.719354] Key type .llcrypt unregistered [10982.721173] Key type ._llcrypt unregistered [11011.864826] Key type ._llcrypt registered [11011.867889] Key type .llcrypt registered [11012.055989] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11013.644380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11013.894555] alg: No test for adler32 (adler32-zlib) [11015.100462] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [11015.291786] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [11016.927315] Key type lgssc registered [11018.017917] Lustre: Echo OBD driver; http://www.lustre.org/ [11025.442849] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11025.462431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11026.837876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11031.131338] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11037.155448] LustreError: 331278:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11037.176378] LustreError: 331278:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11037.898363] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11042.276237] LustreError: 331277:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11043.673605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11044.051609] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11048.812535] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11057.888730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11058.166730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11063.759785] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11068.401334] LustreError: 332967:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11069.996567] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11072.506790] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11072.510270] LustreError: 332967:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11072.565506] LustreError: 332967:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11076.092820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11076.424204] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11081.737921] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11082.144761] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11088.661726] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11095.770590] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11095.970876] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11194.339039] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11194.342977] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11194.348657] LustreError: Skipped 1 previous similar message [11194.352896] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11194.361297] Lustre: Skipped 1 previous similar message [11194.379462] Lustre: Skipped 1 previous similar message [11199.466940] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11199.476826] Lustre: Skipped 1 previous similar message [11200.422864] Lustre: server umount lustre-OST0001 complete [11204.584877] LustreError: 332970:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11204.591366] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11204.595043] LustreError: 332970:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11204.596033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11204.602117] Lustre: Skipped 1 previous similar message [11209.704789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11209.720448] Lustre: Skipped 2 previous similar messages [11210.675827] Lustre: server umount lustre-OST0000 complete [11214.818495] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11214.824864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11214.829446] Lustre: Skipped 1 previous similar message [11214.839087] Lustre: Skipped 1 previous similar message [11219.801109] Lustre: server umount lustre-MDT0000 complete [11219.943126] LustreError: 331274:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11219.967256] LustreError: 331274:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11222.517511] LustreError: 334975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790957055 with bad export cookie 1204094954838694501 [11222.528479] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11222.790319] Lustre: server umount lustre-MDT0001 complete [11230.663306] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [11232.766861] Key type lgssc unregistered [11232.974862] LNet: 336227:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11232.986816] LNetError: 336227:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11234.021958] LNet: Removed LNI 192.168.201.139@tcp [11234.835733] Key type .llcrypt unregistered [11234.838197] Key type ._llcrypt unregistered [11269.086381] Key type ._llcrypt registered [11269.088885] Key type .llcrypt registered [11269.170114] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11269.931218] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11269.946205] alg: No test for adler32 (adler32-zlib) [11270.932450] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [11271.090548] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [11272.736186] Key type lgssc registered [11273.707569] Lustre: Echo OBD driver; http://www.lustre.org/ [11281.010221] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11281.030353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11282.298353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11285.211405] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11291.969612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11292.206596] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11295.504170] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11297.998085] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11303.120761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11303.456255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11308.579440] LustreError: 340000:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11308.587915] LustreError: 340000:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11308.590415] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11308.641136] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11313.639031] LustreError: 340000:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11315.282649] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11315.454570] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11317.507106] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11320.271573] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11327.949649] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11332.541613] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11341.284771] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11341.292075] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11341.296693] Lustre: Skipped 2 previous similar messages [11341.318930] Lustre: Skipped 3 previous similar messages [11346.403122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11346.410187] Lustre: Skipped 3 previous similar messages [11347.348645] Lustre: server umount lustre-MDT0000 complete [11350.058733] LustreError: 338491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790957182 with bad export cookie 15227081269283620238 [11350.064563] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11350.068845] LustreError: 338491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11350.311241] Lustre: server umount lustre-MDT0001 complete [11362.242503] Lustre: server umount lustre-OST0000 complete [11374.439573] Lustre: server umount lustre-OST0001 complete [11384.576615] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11391.775977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11392.078077] LustreError: 343772:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11392.132586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11395.079152] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11397.604340] LustreError: 343773:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11400.982195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11401.223515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11403.842254] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11405.564690] Lustre: 344914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11409.871522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11411.184807] LustreError: 345267:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11414.585810] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11417.581344] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11421.085027] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11421.267292] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11421.275201] Lustre: Skipped 1 previous similar message [11425.475714] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11426.288993] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11431.490821] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11434.459869] Lustre: 346780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11437.026399] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11437.030099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11437.037373] Lustre: Skipped 1 previous similar message [11437.044335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11441.640198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11441.644205] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11441.648215] Lustre: Skipped 1 previous similar message [11441.665549] Lustre: Skipped 1 previous similar message [11443.210506] Lustre: server umount lustre-MDT0000 complete [11445.866470] LustreError: 343754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790957278 with bad export cookie 15227081269283621939 [11445.903832] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11446.175931] Lustre: server umount lustre-MDT0001 complete [11459.560729] Lustre: server umount lustre-OST0000 complete [11472.710903] Lustre: server umount lustre-OST0001 complete [11475.015815] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11476.060338] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11477.042743] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:08:28 (1790957308) [11491.582950] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11497.956843] loop0: detected capacity change from 0 to 1099511627776 [11498.268879] SGI XFS with ACLs, security attributes, quota, no debug enabled [11498.273347] *** VALIDATE xfs *** [11498.284706] XFS (loop0): Mounting V5 Filesystem [11498.304076] XFS (loop0): Ending clean mount [11499.114318] loop1: detected capacity change from 0 to 18691697672192 [11556.114866] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [11562.596298] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11583.425619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11583.632846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11583.651815] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11583.701833] Lustre: lustre-MDT0000: new disk, initializing [11583.764754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11583.778834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11587.034402] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11594.215164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11594.312459] Lustre: 351902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11594.353684] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11594.357194] Lustre: Skipped 1 previous similar message [11594.403736] Lustre: lustre-MDT0001: new disk, initializing [11594.493629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11594.507885] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11597.073466] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11603.450638] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11607.284322] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11611.851848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11612.041636] Lustre: lustre-OST0000: new disk, initializing [11612.044185] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11612.050609] Lustre: 353868:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11613.807246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11613.811924] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11613.897582] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11616.007406] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11621.783929] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11624.338243] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11624.484823] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11626.914822] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11627.062960] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11629.543726] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11629.549022] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11629.550171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11629.550177] Lustre: Skipped 2 previous similar messages [11629.576539] Lustre: Skipped 2 previous similar messages [11634.360093] Lustre: server umount lustre-OST0000 complete [11639.778840] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11639.783706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11639.789896] Lustre: Skipped 1 previous similar message [11639.794184] Lustre: Skipped 1 previous similar message [11643.319145] Lustre: server umount lustre-MDT0000 complete [11644.897379] LustreError: 351913:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11644.920342] LustreError: 351913:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [11645.739988] LustreError: 351894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790957478 with bad export cookie 15227081269283623486 [11645.746536] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11645.751910] LustreError: 351894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [11645.970601] Lustre: server umount lustre-MDT0001 complete [11650.759940] XFS (loop0): Unmounting Filesystem [11653.937487] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:11:25 (1790957485) [11658.328550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11658.693540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11658.700341] Lustre: Skipped 2 previous similar messages [11661.565150] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11663.843032] LustreError: 356421:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11663.859782] LustreError: 356421:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11667.735943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11670.802530] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11675.868476] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11679.738729] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11684.717092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11689.226158] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11693.766852] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11712.992248] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11712.994834] LustreError: Skipped 1 previous similar message [11712.997945] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11713.004836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11724.255361] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11724.383527] Lustre: server umount lustre-OST0000 complete [11729.380227] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11729.382865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11729.388459] Lustre: Skipped 2 previous similar messages [11729.394045] LustreError: Skipped 1 previous similar message [11741.151156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11741.275764] Lustre: server umount lustre-MDT0000 complete [11743.396822] LustreError: 356403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790957576 with bad export cookie 15227081269283624368 [11743.405892] LustreError: 356403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11743.407630] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11743.616399] Lustre: server umount lustre-MDT0001 complete [11750.042815] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [11751.872486] Key type lgssc unregistered [11752.073711] LNet: 360066:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11752.077702] LNetError: 360066:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11753.125773] LNet: Removed LNI 192.168.201.139@tcp [11753.749266] Key type .llcrypt unregistered [11753.751622] Key type ._llcrypt unregistered [11773.610336] Key type ._llcrypt registered [11773.612573] Key type .llcrypt registered [11773.713945] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [11774.501120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11774.564802] alg: No test for adler32 (adler32-zlib) [11775.651806] Lustre: Lustre: Build Version: 2.17.59_3_g5d853fc [11775.818789] LNet: Added LNI 192.168.201.139@tcp [8/256/0/180] [11777.455208] Key type lgssc registered [11778.326849] Lustre: Echo OBD driver; http://www.lustre.org/ [11783.882559] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11783.902361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11785.238035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11788.315925] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11794.434235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11794.744720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11798.091431] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11802.906693] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11806.438850] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11811.189305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11811.396396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11815.381386] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11820.410383] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11836.280560] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.39@tcp (stopping) [11836.385466] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11841.300985] Lustre: server umount lustre-MDT0001 complete [11841.368487] LustreError: 361625:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11841.385551] LustreError: 361625:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11846.490507] LustreError: 361623:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11846.512849] LustreError: 361623:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11849.521288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11849.586240] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11849.589466] Lustre: Found index 1 for lustre-MDT0001, updating log [11849.617695] Lustre: 361617:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11849.906406] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11853.201894] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [11855.338948] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11858.951041] LustreError: 365296:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11858.955463] LustreError: 365296:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11858.960094] LustreError: 361620:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11859.000671] LustreError: 365299:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.139@tcp: cfg command failed: rc = -17 [11859.009100] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11859.009100] [12157.409490] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12157.420307] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12157.426657] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12157.431670] Lustre: Skipped 2 previous similar messages [12157.436489] Lustre: Skipped 2 previous similar messages [12159.343029] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.39@tcp (stopping) [12159.352684] Lustre: Skipped 1 previous similar message [12162.531717] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12162.539689] Lustre: Skipped 1 previous similar message [12163.560774] Lustre: server umount lustre-MDT0001 complete [12164.443606] LustreError: 361624:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.39@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12164.456993] LustreError: 361624:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12167.653739] LustreError: 362355:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12167.663789] LustreError: 362355:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12172.777482] LustreError: 361623:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12172.799069] LustreError: 361623:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12173.441255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12173.599187] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12173.608455] Lustre: Found index 1 for lustre-MDT0001, updating log [12173.677726] Lustre: 361616:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12173.989568] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12178.464704] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [12179.444737] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12182.511853] LustreError: 373823:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12182.519177] LustreError: 373823:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12182.526099] LustreError: 361620:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12182.528298] LustreError: 361620:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12182.614875] LustreError: 373833:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12182.625029] LustreError: 373833:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12182.632264] LustreError: 373833:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.139@tcp: cfg command failed: rc = -17 [12182.645283] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.139@tcp [12182.645283] [12486.624441] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12486.634667] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12486.636536] Lustre: Skipped 1 previous similar message [12486.637044] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12486.645419] Lustre: Skipped 1 previous similar message [12490.247367] Lustre: server umount lustre-MDT0001 complete [12491.751131] LustreError: 369295:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12491.776394] LustreError: 369295:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12500.679232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12500.819419] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12500.822031] Lustre: Found index 1 for lustre-MDT0001, updating log [12500.880205] Lustre: 361618:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12501.219656] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12505.851240] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [12506.605910] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12507.143951] LustreError: 382521:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12507.156326] LustreError: 382521:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12507.169247] LustreError: 361620:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12507.173779] LustreError: 361620:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12507.231910] LustreError: 382527:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12507.241374] LustreError: 382527:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.139@tcp: cfg command failed: rc = -114 [12507.247416] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12507.247416] [12823.521152] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12823.525827] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12823.533423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12823.537139] Lustre: Skipped 1 previous similar message [12829.620389] Lustre: server umount lustre-MDT0000 complete [12832.198696] LustreError: 382179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790958665 with bad export cookie 1884696148100351020 [12832.202251] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12832.207948] LustreError: 382179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12834.273321] LustreError: 367526:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12834.274760] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12834.285317] LustreError: 367526:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12834.307366] Lustre: Skipped 2 previous similar messages [12838.733630] Lustre: server umount lustre-MDT0001 complete [12842.072981] Lustre: server umount lustre-OST0000 complete [12848.800368] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [12855.102228] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [12881.604870] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:31:52 (1790958712) [12886.532345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12886.715847] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12886.735662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12886.791617] Lustre: lustre-MDT0000: new disk, initializing [12886.844543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12886.862602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12890.051117] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [12898.129689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12898.211208] Lustre: 394123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12898.257492] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12898.261308] Lustre: Skipped 1 previous similar message [12898.332982] Lustre: lustre-MDT0001: new disk, initializing [12898.383347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12898.412896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12898.429080] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12901.770495] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [12908.977217] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12913.450756] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12919.434570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12919.649396] Lustre: lustre-OST0000: new disk, initializing [12919.655790] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12919.663266] Lustre: 396087:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12919.744459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12924.955502] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_default_debug -1 all [12928.569203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12928.583191] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12928.677471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12931.512779] Lustre: DEBUG MARKER: oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12939.236218] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12939.241906] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12939.242770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12939.245129] Lustre: Skipped 8 previous similar messages [12939.285604] Lustre: Skipped 2 previous similar messages [12943.287337] Lustre: server umount lustre-MDT0000 complete [12944.354292] LustreError: 394128:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12944.371279] LustreError: 394128:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12945.738856] LustreError: 394915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790958778 with bad export cookie 1884696148100352973 [12945.740180] LustreError: 394927:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12945.742622] LustreError: MGC192.168.201.139@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12945.748058] LustreError: 394915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12946.021742] Lustre: server umount lustre-MDT0001 complete [12959.115587] Lustre: server umount lustre-OST0000 complete [12979.855882] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing set_hostid [12986.086959] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing load_modules_local [13008.451256] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12672 sec ========= 12:33:59 (1790958839) [13009.870658] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:34:01 (1790958841) === [13012.627489] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:34:03 (1790958843) === [13044.004229] Lustre: DEBUG MARKER: oleg139-server.virtnet: executing unload_modules_local [13045.927732] Key type lgssc unregistered [13046.124502] LNet: 402446:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13046.131757] LNetError: 402446:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13046.141958] LNet: Removed LNI 192.168.201.139@tcp [13046.812237] Key type .llcrypt unregistered [13046.814912] Key type ._llcrypt unregistered