[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 497084014 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002381] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010123] LSM: Security Framework initializing [ 0.012040] Yama: becoming mindful. [ 0.013048] SELinux: Initializing. [ 0.014090] *** VALIDATE selinux *** [ 0.023184] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028367] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029192] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030113] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031124] *** VALIDATE tmpfs *** [ 0.033278] *** VALIDATE proc *** [ 0.034273] *** VALIDATE cgroup *** [ 0.035011] *** VALIDATE cgroup2 *** [ 0.036247] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038089] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040031] Spectre V2 : User space: Vulnerable [ 0.041010] Speculative Store Bypass: Vulnerable [ 0.044563] debug: unmapping init [mem 0xffffffffb4659000-0xffffffffb4660fff] [ 0.046185] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047734] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048030] ... version: 2 [ 0.049016] ... bit width: 48 [ 0.050011] ... generic registers: 4 [ 0.051017] ... value mask: 0000ffffffffffff [ 0.052015] ... max period: 00007fffffffffff [ 0.053022] ... fixed-purpose events: 3 [ 0.054016] ... event mask: 000000070000000f [ 0.055363] rcu: Hierarchical SRCU implementation. [ 0.057587] smp: Bringing up secondary CPUs ... [ 0.058626] x86: Booting SMP configuration: [ 0.059041] .... node #0, CPUs: #1 #2 #3 [ 0.063017] smp: Brought up 1 node, 4 CPUs [ 0.065020] smpboot: Max logical packages: 1 [ 0.066013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.099018] node 0 deferred pages initialised in 30ms [ 0.103107] devtmpfs: initialized [ 0.104205] x86/mm: Memory block size: 128MB [ 0.106255] gcov: version magic: 0x41383552 [ 0.108253] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.110071] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.111272] pinctrl core: initialized pinctrl subsystem [ 0.113124] [ 0.113602] ************************************************************* [ 0.115010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.117008] ** ** [ 0.118009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.120008] ** ** [ 0.121006] ** This means that this kernel is built to expose internal ** [ 0.123013] ** IOMMU data structures, which may compromise security on ** [ 0.124008] ** your system. ** [ 0.126011] ** ** [ 0.127007] ** If you see this message and you are not debugging the ** [ 0.129009] ** kernel, report this immediately to your vendor! ** [ 0.130008] ** ** [ 0.132007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.133007] ************************************************************* [ 0.134578] NET: Registered protocol family 16 [ 0.136288] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.138043] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.139047] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.142097] cpuidle: using governor menu [ 0.143404] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.145455] PCI: Using configuration type 1 for base access [ 0.147106] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.154044] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.155056] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.158133] cryptd: max_cpu_qlen set to 1000 [ 0.161192] ACPI: Added _OSI(Module Device) [ 0.162012] ACPI: Added _OSI(Processor Device) [ 0.163013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.165017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.170074] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.175385] ACPI: Interpreter enabled [ 0.177061] ACPI: PM: (supports S0 S3 S4 S5) [ 0.178015] ACPI: Using IOAPIC for interrupt routing [ 0.180109] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.184395] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.193701] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.196036] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.198023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.202083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.207375] acpiphp: Slot [2] registered [ 0.208106] acpiphp: Slot [5] registered [ 0.210127] acpiphp: Slot [6] registered [ 0.211121] acpiphp: Slot [7] registered [ 0.213125] acpiphp: Slot [8] registered [ 0.215120] acpiphp: Slot [9] registered [ 0.216126] acpiphp: Slot [10] registered [ 0.218141] acpiphp: Slot [3] registered [ 0.219099] acpiphp: Slot [4] registered [ 0.221174] acpiphp: Slot [11] registered [ 0.224156] acpiphp: Slot [12] registered [ 0.226119] acpiphp: Slot [13] registered [ 0.228143] acpiphp: Slot [14] registered [ 0.230120] acpiphp: Slot [15] registered [ 0.232129] acpiphp: Slot [16] registered [ 0.235146] acpiphp: Slot [17] registered [ 0.236084] acpiphp: Slot [18] registered [ 0.237088] acpiphp: Slot [19] registered [ 0.238087] acpiphp: Slot [20] registered [ 0.239096] acpiphp: Slot [21] registered [ 0.240159] acpiphp: Slot [22] registered [ 0.242105] acpiphp: Slot [23] registered [ 0.243078] acpiphp: Slot [24] registered [ 0.244062] acpiphp: Slot [25] registered [ 0.245058] acpiphp: Slot [26] registered [ 0.246000] acpiphp: Slot [27] registered [ 0.246000] acpiphp: Slot [28] registered [ 0.247084] acpiphp: Slot [29] registered [ 0.248116] acpiphp: Slot [30] registered [ 0.249093] acpiphp: Slot [31] registered [ 0.251092] PCI host bridge to bus 0000:00 [ 0.252017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.253025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.255019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.256020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.258019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.260021] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.262161] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.263876] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.267360] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.278015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.283057] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.286070] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.289019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.291026] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.293316] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.297826] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.301047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.305781] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.312017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.327931] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.333925] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.339610] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.344016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.349019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.361015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.368943] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.376019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.383017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.399026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.410000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.415018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.422020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.433017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.443756] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.451017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.458020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.488026] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.502009] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.513023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.523019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.548020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.559376] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.567022] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.574015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.591016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.602245] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.604394] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.607375] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.610675] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.613252] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.617033] iommu: Default domain type: Passthrough [ 0.619479] SCSI subsystem initialized [ 0.621146] ACPI: bus type USB registered [ 0.623136] usbcore: registered new interface driver usbfs [ 0.625165] usbcore: registered new interface driver hub [ 0.627096] usbcore: registered new device driver usb [ 0.629128] pps_core: LinuxPPS API ver. 1 registered [ 0.632015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.636070] PTP clock support registered [ 0.639066] EDAC MC: Ver: 3.0.0 [ 0.641142] PCI: Using ACPI for IRQ routing [ 0.642553] NetLabel: Initializing [ 0.644016] NetLabel: domain hash size = 128 [ 0.645010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.647087] NetLabel: unlabeled traffic allowed by default [ 0.650154] vgaarb: loaded [ 0.651291] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.654014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.661330] clocksource: Switched to clocksource kvm-clock [ 0.771057] VFS: Disk quotas dquot_6.6.0 [ 0.773290] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.777665] *** VALIDATE ramfs *** [ 0.779578] *** VALIDATE hugetlbfs *** [ 0.781722] pnp: PnP ACPI init [ 0.784556] pnp: PnP ACPI: found 6 devices [ 0.832055] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.834728] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.836801] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.838556] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.840286] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.841976] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.844759] NET: Registered protocol family 2 [ 0.847322] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.852083] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.855835] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.862954] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.867543] TCP: Hash tables configured (established 65536 bind 65536) [ 0.871766] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.876485] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.880358] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.884824] NET: Registered protocol family 1 [ 0.890274] RPC: Registered named UNIX socket transport module. [ 0.892572] RPC: Registered udp transport module. [ 0.894298] RPC: Registered tcp transport module. [ 0.895899] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.898527] NET: Registered protocol family 44 [ 0.900279] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.902783] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.905071] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.907623] PCI: CLS 0 bytes, default 64 [ 0.909545] Unpacking initramfs... [ 2.315972] debug: unmapping init [mem 0xffff8972bcc54000-0xffff8972bffbffff] [ 2.319445] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.321828] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.324775] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.816432] Initialise system trusted keyrings [ 2.818232] Key type blacklist registered [ 2.820382] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.829862] zbud: loaded [ 2.833501] *** VALIDATE nfs *** [ 2.835065] *** VALIDATE nfs4 *** [ 2.837284] pstore: using deflate compression [ 2.843861] Platform Keyring initialized [ 2.948927] NET: Registered protocol family 38 [ 2.950921] Key type asymmetric registered [ 2.952655] Asymmetric key parser 'x509' registered [ 2.955021] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.958318] io scheduler mq-deadline registered [ 2.960466] io scheduler kyber registered [ 2.962340] io scheduler bfq registered [ 2.964543] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.967650] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.971105] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.974454] ACPI: Power Button [PWRF] [ 2.980709] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.988521] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.001717] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.008256] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.020920] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.048736] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.076638] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.082095] Non-volatile memory driver v1.3 [ 3.083725] Linux agpgart interface v0.103 [ 3.116484] virtio_blk virtio1: [vda] 141800 512-byte logical blocks (72.6 MB/69.2 MiB) [ 3.119766] vda: detected capacity change from 0 to 72601600 [ 3.134310] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.137452] vdb: detected capacity change from 0 to 1073741824 [ 3.152435] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.155475] vdc: detected capacity change from 0 to 2621440000 [ 3.169454] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.173706] vdd: detected capacity change from 0 to 2621440000 [ 3.187450] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.190564] vde: detected capacity change from 0 to 4294967296 [ 3.204766] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.207658] vdf: detected capacity change from 0 to 4294967296 [ 3.214173] libphy: Fixed MDIO Bus: probed [ 3.228299] usbcore: registered new interface driver usbserial_generic [ 3.231500] usbserial: USB Serial support registered for generic [ 3.234396] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.239457] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.241824] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.244876] mousedev: PS/2 mouse device common for all mice [ 3.248476] rtc_cmos 00:05: RTC can wake from S4 [ 3.252000] rtc_cmos 00:05: registered as rtc0 [ 3.254137] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.256330] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.258559] intel_pstate: CPU model not supported [ 3.264926] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.270507] hid: raw HID events driver (C) Jiri Kosina [ 3.271567] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.274329] usbcore: registered new interface driver usbhid [ 3.279509] usbhid: USB HID core driver [ 3.281624] drop_monitor: Initializing network drop monitor service [ 3.284501] Initializing XFRM netlink socket [ 3.286376] NET: Registered protocol family 10 [ 3.289135] Segment Routing with IPv6 [ 3.290841] NET: Registered protocol family 17 [ 3.292323] mpls_gso: MPLS GSO support [ 3.298960] RAS: Correctable Errors collector initialized. [ 3.301195] AVX version of gcm_enc/dec engaged. [ 3.303053] AES CTR mode by8 optimization enabled [ 3.376476] sched_clock: Marking stable (3376448679, 0)->(4302146594, -925697915) [ 3.380424] registered taskstats version 1 [ 3.382397] Loading compiled-in X.509 certificates [ 3.384543] zswap: loaded using pool lzo/zbud [ 3.406995] Key type big_key registered [ 3.418461] Key type encrypted registered [ 3.420252] ima: No TPM chip found, activating TPM-bypass! [ 3.422251] ima: Allocated hash algorithm: sha1 [ 3.424124] ima: No architecture policies found [ 3.425930] evm: Initialising EVM extended attributes: [ 3.427862] evm: security.selinux [ 3.429237] evm: security.ima [ 3.430459] evm: security.capability [ 3.431880] evm: HMAC attrs: 0x1 [ 3.434353] rtc_cmos 00:05: setting system clock to 2026-05-18 17:45:16 UTC (1779126316) [ 3.441429] debug: unmapping init [mem 0xffffffffb5603000-0xffffffffb57fffff] [ 3.445156] debug: unmapping init [mem 0xffffffffb4382000-0xffffffffb4658fff] [ 3.457109] Write protecting the kernel read-only data: 28672k [ 3.459851] debug: unmapping init [mem 0xffffffffb2a03000-0xffffffffb2bfffff] [ 3.462502] debug: unmapping init [mem 0xffffffffb3314000-0xffffffffb33fffff] [ 3.492963] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.500105] systemd[1]: Detected virtualization kvm. [ 3.501851] systemd[1]: Detected architecture x86-64. [ 3.503148] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.533420] systemd[1]: No hostname configured. [ 3.535558] systemd[1]: Set hostname to . [ 3.538327] random: systemd: uninitialized urandom read (16 bytes read) [ 3.541789] systemd[1]: Initializing machine ID from random generator. [ 3.594276] random: ln: uninitialized urandom read (6 bytes read) [ 3.687876] random: systemd: uninitialized urandom read (16 bytes read) [ 3.692301] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.699048] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.706115] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Reached target Timers. Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Control Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.212100] device-mapper: uevent: version 1.0.3 [ 4.213956] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK [ 4.500458] random: fast init done ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.822036] virtio_net virtio0 ens2: renamed from eth0 [ 4.883000] scsi host0: ata_piix [ 4.915035] scsi host1: ata_piix [ 4.917076] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.920471] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.060246] dracut-initqueue[588]: RTNETLINK answers: File exists [ 9.946182] random: crng init done [ 9.947333] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.149965] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.216303] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.449671] SELinux: Disabled at runtime. [ 11.509177] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.517588] systemd[1]: Detected virtualization kvm. [ 11.519332] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.965226] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.968207] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.976139] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.979924] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.983396] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.989752] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.994179] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ 12.074575] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target RPC Port Mapper. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.433753] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.726613] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.762628] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.838225] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.867346] EDAC sbridge: Ver: 1.1.2 [ 14.362743] Key type dns_resolver registered [ 14.664545] NFS: Registering the id_resolver key type [ 14.666418] Key type id_resolver registered [ 14.667910] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Login Service. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg141-server login: [ 42.227794] libcfs: loading out-of-tree module taints kernel. [ 42.251830] Key type ._llcrypt registered [ 42.253193] Key type .llcrypt registered [ 42.321802] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 66.133822] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 67.888594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 67.910386] alg: No test for adler32 (adler32-zlib) [ 69.228960] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 70.065963] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 71.880308] Key type lgssc registered [ 73.924582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 80.081121] hrtimer: interrupt took 9120984 ns [ 92.474996] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 139.091725] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 154.059396] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 154.090167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 155.367423] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 155.411790] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 155.573201] Lustre: lustre-MDT0000: new disk, initializing [ 155.701454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 155.731098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 160.380508] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 175.475602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 175.617274] Lustre: 6544:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 175.661502] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 175.666026] Lustre: Skipped 1 previous similar message [ 175.838765] Lustre: lustre-MDT0001: new disk, initializing [ 175.955848] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 175.993619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 176.011378] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 181.649451] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 187.380278] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 198.633065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 198.925106] Lustre: lustre-OST0000: new disk, initializing [ 198.928478] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 198.936409] Lustre: 8449:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 198.984973] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 204.339387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 204.350681] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 204.400272] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 205.951198] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 221.982828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 222.083195] Lustre: lustre-OST0001: new disk, initializing [ 222.089528] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 222.093890] Lustre: 9503:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 222.163960] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 227.431911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 227.437675] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 227.551247] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 229.472460] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 242.781793] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 249.350444] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 256.746655] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing check_logdir /tmp/testlogs/ [ 263.024672] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing yml_node [ 269.767659] Lustre: DEBUG MARKER: Client: 2.17.53.24 [ 272.850828] Lustre: DEBUG MARKER: MDS: 2.17.53.24 [ 275.676343] Lustre: DEBUG MARKER: OSS: 2.17.53.24 [ 277.923333] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon May 18 13:49:49 EDT 2026 [ 299.236122] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 301.310678] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 319.459083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 319.472930] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.486540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 319.975826] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.994317] Lustre: Skipped 1 previous similar message [ 320.003499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 320.012336] Lustre: Skipped 2 previous similar messages [ 324.627603] Lustre: server umount lustre-MDT0000 complete [ 325.121301] LustreError: 9072:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 325.152558] LustreError: 9072:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 329.249066] LustreError: 6534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779126642 with bad export cookie 12375384831115164833 [ 329.263760] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 329.269128] LustreError: 6534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 330.213469] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 330.216211] LustreError: 6550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.221215] Lustre: Skipped 1 previous similar message [ 330.222066] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 330.263682] LustreError: 6550:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 334.241323] LustreError: 9072:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 334.250128] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 334.285561] LustreError: 9072:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 334.327432] Lustre: Skipped 2 previous similar messages [ 335.880646] Lustre: server umount lustre-MDT0001 complete [ 342.267482] Lustre: server umount lustre-OST0000 complete [ 347.746615] Lustre: server umount lustre-OST0001 complete [ 356.369992] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 367.276421] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 411.882983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 412.287394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 412.343526] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 412.469720] Lustre: lustre-MDT0000: new disk, initializing [ 412.639864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 412.691603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 418.774044] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 433.226850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 433.430594] Lustre: 15033:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 433.511229] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 433.524147] Lustre: Skipped 1 previous similar message [ 433.590271] Lustre: lustre-MDT0001: new disk, initializing [ 433.693378] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 433.750937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 433.763135] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 438.751482] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 447.179268] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 452.123755] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 461.789961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 462.256351] Lustre: lustre-OST0000: new disk, initializing [ 462.260302] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 462.265045] Lustre: 16775:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 462.421938] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 463.849174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 463.862343] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 463.946540] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 469.206520] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 477.484722] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 483.039806] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 483.279610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 488.502721] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 488.783043] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 494.572605] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 494.579763] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 494.589841] Lustre: Skipped 1 previous similar message [ 494.602560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 497.582943] Lustre: server umount lustre-OST0000 complete [ 504.809883] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 504.823898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 504.826764] Lustre: Skipped 2 previous similar messages [ 504.841084] Lustre: Skipped 2 previous similar messages [ 508.273881] Lustre: server umount lustre-MDT0000 complete [ 509.921858] LustreError: 15887:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.953324] LustreError: 15887:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 512.570351] LustreError: 15025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779126825 with bad export cookie 12375384831115166366 [ 512.573342] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 512.583547] LustreError: 15025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 513.039861] Lustre: server umount lustre-MDT0001 complete [ 519.733885] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 13:53:51 (1779126831) [ 527.835059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 528.563361] LustreError: 18894:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.671740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 534.391916] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 538.083976] LustreError: 18895:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 538.102114] LustreError: 18895:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 544.122253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 550.126640] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 557.018996] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 561.532319] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 570.058357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 577.504292] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 584.495588] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 591.853936] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 599.523536] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 599.538520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 599.540818] Lustre: Skipped 1 previous similar message [ 599.572213] Lustre: Skipped 2 previous similar messages [ 606.100150] Lustre: server umount lustre-MDT0000 complete [ 609.764175] LustreError: 18895:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 609.785613] LustreError: 18895:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 610.319280] LustreError: 18874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779126923 with bad export cookie 12375384831115167248 [ 610.325951] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 610.330288] LustreError: 18874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 610.737822] Lustre: server umount lustre-MDT0001 complete [ 625.239942] Lustre: server umount lustre-OST0000 complete [ 651.677077] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 664.653372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 665.328553] LustreError: 22931:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 665.369381] LustreError: 22931:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 665.476631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 665.489317] Lustre: Skipped 2 previous similar messages [ 672.074434] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 684.060645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 689.478173] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 692.916801] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 701.698732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 710.595510] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 721.814578] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 722.097711] Lustre: lustre-OST0001: new disk, initializing [ 722.109533] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 722.128193] Lustre: 25201:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 723.576691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 723.599529] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 723.776849] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 729.682559] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 741.373806] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 746.403141] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 759.780219] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 759.784109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 759.795441] Lustre: Skipped 3 previous similar messages [ 759.813440] Lustre: Skipped 6 previous similar messages [ 764.727734] Lustre: server umount lustre-MDT0000 complete [ 764.900169] LustreError: 22926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 764.922829] LustreError: 22926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 769.111050] LustreError: 22912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127082 with bad export cookie 12375384831115168095 [ 769.113929] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 769.127129] LustreError: 22912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 769.760683] Lustre: server umount lustre-MDT0001 complete [ 784.366581] Lustre: server umount lustre-OST0000 complete [ 786.297959] Lustre: 3654:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779127083/real 1779127083] req@ffff89720157c000 x1865549226432896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779127099 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 788.772806] Lustre: server umount lustre-OST0001 complete [ 807.084707] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 821.401273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 822.413077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 822.426587] Lustre: Skipped 3 previous similar messages [ 827.330664] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 838.244887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 842.956835] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 846.588044] Lustre: 29259:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 857.448148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 867.255422] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 879.812934] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 887.275188] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 896.700165] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 901.373892] Lustre: 31107:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 910.309989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 910.325392] LustreError: Skipped 1 previous similar message [ 910.334228] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 910.358105] Lustre: Skipped 2 previous similar messages [ 910.367359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 915.971616] Lustre: server umount lustre-MDT0000 complete [ 919.549735] LustreError: 29134:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 919.613628] LustreError: 29134:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 920.609725] LustreError: 29261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127233 with bad export cookie 12375384831115169383 [ 920.622251] LustreError: 29261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 920.626250] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 921.221300] Lustre: server umount lustre-MDT0001 complete [ 936.906210] Lustre: server umount lustre-OST0000 complete [ 940.000183] Lustre: 3656:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779127237/real 1779127237] req@ffff897202563b80 x1865549226491136/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779127253 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 941.024150] Lustre: 3653:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779127238/real 1779127238] req@ffff89732f675880 x1865549226491392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779127254 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 941.367694] Lustre: server umount lustre-OST0001 complete [ 952.581048] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:01:03 (1779127263) [ 989.973480] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1001.025248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1007.108614] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1020.069858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1028.026123] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1032.413484] Lustre: 35054:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1040.546790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1048.152233] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1057.473030] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1064.733357] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1074.942414] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1079.903199] Lustre: 36900:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1087.467578] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1087.471671] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1087.471978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1087.471984] Lustre: Skipped 4 previous similar messages [ 1087.478403] Lustre: Skipped 7 previous similar messages [ 1093.221968] Lustre: server umount lustre-MDT0000 complete [ 1098.580831] LustreError: 33919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127411 with bad export cookie 12375384831115170496 [ 1098.587976] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1098.593743] LustreError: 33919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1099.161174] Lustre: server umount lustre-MDT0001 complete [ 1115.473685] Lustre: server umount lustre-OST0000 complete [ 1118.696338] Lustre: 3653:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779127415/real 1779127415] req@ffff8973037b3100 x1865549226547840/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779127431 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1119.925935] Lustre: server umount lustre-OST0001 complete [ 1131.500897] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:04:02 (1779127442) [ 1156.402874] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1171.829607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1172.340237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1172.345776] Lustre: Skipped 7 previous similar messages [ 1177.188954] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1177.570629] LustreError: 39047:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1177.577982] LustreError: 39047:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 16 previous similar messages [ 1187.139564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1193.193888] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1197.213157] Lustre: 40164:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1206.050237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1213.574364] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1225.056126] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1233.366301] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1243.861452] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1247.934342] Lustre: 42007:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1249.159479] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1249.163623] Lustre: Skipped 1 previous similar message [ 1254.372968] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1254.382800] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1254.409151] Lustre: Skipped 1 previous similar message [ 1259.144834] Lustre: server umount lustre-MDT0000 complete [ 1264.090568] LustreError: 39783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127577 with bad export cookie 12375384831115171595 [ 1264.099356] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1264.104389] LustreError: 39783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1270.767972] Lustre: server umount lustre-MDT0001 complete [ 1281.891244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1289.185836] LustreError: 3652:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff89730b533800 x1865549226604288/t0(0) o250->MGC192.168.201.141@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1294.624343] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1299.949685] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1304.391492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1310.220729] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1310.238943] Lustre: Skipped 1 previous similar message [ 1310.253275] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1317.148912] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1319.075814] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1323.234886] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1324.832907] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1336.826189] Lustre: server umount lustre-MDT0000 complete [ 1341.697541] LustreError: 39783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127654 with bad export cookie 12375384831115172694 [ 1341.705064] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1341.711665] LustreError: 39783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1342.027657] Lustre: server umount lustre-MDT0001 complete [ 1358.449802] Lustre: server umount lustre-OST0000 complete [ 1362.401070] Lustre: 3655:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779127659/real 1779127659] req@ffff89732dd47b80 x1865549226638592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779127675 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1362.412814] Lustre: 3655:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1363.602898] Lustre: server umount lustre-OST0001 complete [ 1375.857257] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:08:07 (1779127687) [ 1394.229981] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1405.805761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1411.503289] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1420.629149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1425.770125] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1428.520497] Lustre: 47502:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1436.117335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1443.965853] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1452.826512] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1460.576585] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1469.183515] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1474.390791] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1492.961554] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1492.980493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1492.991440] Lustre: Skipped 26 previous similar messages [ 1495.348125] Lustre: server umount lustre-MDT0000 complete [ 1499.523826] LustreError: 46367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779127812 with bad export cookie 12375384831115173520 [ 1499.527647] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1499.553570] LustreError: 46367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1500.210238] Lustre: server umount lustre-MDT0001 complete [ 1514.878691] Lustre: server umount lustre-OST0000 complete [ 1529.882168] Lustre: server umount lustre-OST0001 complete [ 1541.584357] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:10:53 (1779127853) [ 1545.084301] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1547.578522] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:10:58 (1779127858) [ 1561.154977] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 1573.353444] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1603.651603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1604.093635] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1604.117363] Lustre: Skipped 1 previous similar message [ 1604.134435] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1604.276960] Lustre: lustre-MDT0000: new disk, initializing [ 1604.419461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1608.485852] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1617.950115] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1626.008412] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1626.286684] Lustre: lustre-OST0000: new disk, initializing [ 1626.295097] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1626.299514] Lustre: Skipped 1 previous similar message [ 1626.308715] Lustre: 53409:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1627.856844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1627.871357] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1627.933139] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1633.457083] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1642.656350] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1645.270336] Lustre: DEBUG MARKER: create test files [ 1684.449821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1684.474385] Lustre: Skipped 14 previous similar messages [ 1687.477108] Lustre: server umount lustre-OST0000 complete [ 1692.935527] Lustre: server umount lustre-MDT0000 complete [ 1706.354865] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1709.492632] Key type lgssc unregistered [ 1709.874948] LNet: 54908:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1709.890190] LNetError: 54908:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1710.950731] LNet: Removed LNI 192.168.201.141@tcp [ 1712.203317] Key type .llcrypt unregistered [ 1712.205695] Key type ._llcrypt unregistered [ 1751.932211] Key type ._llcrypt registered [ 1751.936825] Key type .llcrypt registered [ 1752.090622] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1753.794310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1753.812839] alg: No test for adler32 (adler32-zlib) [ 1755.075277] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 1755.434481] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1757.152870] Key type lgssc registered [ 1759.302358] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1770.384046] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1770.409633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1772.048988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1778.494911] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1786.219629] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1794.939716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1795.450737] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1797.496076] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1797.636759] Lustre: 57568:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1797.650027] Lustre: 57568:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1797.659482] Lustre: 57568:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1797.671125] Lustre: 57568:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1797.681438] Lustre: 57568:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1797.693111] Lustre: 57568:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1802.593874] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1810.303554] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1816.202700] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1823.376408] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1833.447834] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1833.456716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1833.521717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1838.576096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1843.693667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1846.240376] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1846.514824] Lustre: server umount lustre-OST0000 complete [ 1853.469573] Lustre: server umount lustre-MDT0000 complete [ 1867.616439] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1870.781535] Key type lgssc unregistered [ 1871.112883] LNet: 59052:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1871.118849] LNetError: 59052:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1871.136757] LNet: Removed LNI 192.168.201.141@tcp [ 1872.067412] Key type .llcrypt unregistered [ 1872.071430] Key type ._llcrypt unregistered [ 1916.597332] Key type ._llcrypt registered [ 1916.599501] Key type .llcrypt registered [ 1916.724528] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1918.342759] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1918.372217] alg: No test for adler32 (adler32-zlib) [ 1919.571408] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 1919.896538] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1921.672418] Key type lgssc registered [ 1923.249045] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1933.888799] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1933.915624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1935.429565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1940.611524] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1946.957124] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1955.901845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1956.358623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1960.456901] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1960.536564] Lustre: 61816:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1960.544440] Lustre: 61816:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1960.556031] Lustre: 61816:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1960.562704] Lustre: 61816:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1960.580055] Lustre: 61816:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1960.592531] Lustre: 61816:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1964.192370] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1970.958751] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1976.643937] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2016.737695] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2016.762354] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2016.793566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2021.667231] Lustre: server umount lustre-OST0000 complete [ 2026.878187] Lustre: server umount lustre-MDT0000 complete [ 2040.438287] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 2043.888973] Key type lgssc unregistered [ 2044.328835] LNet: 63276:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2044.338314] LNetError: 63276:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2045.352071] LNet: Removed LNI 192.168.201.141@tcp [ 2046.448248] Key type .llcrypt unregistered [ 2046.452581] Key type ._llcrypt unregistered [ 2070.056781] Key type ._llcrypt registered [ 2070.061968] Key type .llcrypt registered [ 2070.199538] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2087.692987] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2089.482351] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2089.516788] alg: No test for adler32 (adler32-zlib) [ 2090.497708] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 2090.651770] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 2092.258145] Key type lgssc registered [ 2093.589318] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2138.848346] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2138.873427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2140.286616] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2140.389836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2140.574869] Lustre: lustre-MDT0000: new disk, initializing [ 2140.857655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2140.894473] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2147.007302] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2158.954933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2159.095500] Lustre: 66451:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2159.156992] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2159.163304] Lustre: Skipped 1 previous similar message [ 2159.260968] Lustre: lustre-MDT0001: new disk, initializing [ 2159.385838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2159.415627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2159.434736] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2164.054897] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2172.687886] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2176.592643] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2185.208136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2185.569885] Lustre: lustre-OST0000: new disk, initializing [ 2185.576420] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2185.591688] Lustre: 68194:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2185.716968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2191.925604] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2193.049835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2193.076649] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2193.188577] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2199.308847] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2204.760796] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2205.015453] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2209.398772] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2209.584437] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2213.857849] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2213.862868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2213.867892] LustreError: Skipped 1 previous similar message [ 2213.872609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2213.881824] Lustre: Skipped 1 previous similar message [ 2213.899598] Lustre: Skipped 1 previous similar message [ 2217.650746] Lustre: server umount lustre-OST0000 complete [ 2224.101135] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2224.106384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2224.118920] Lustre: Skipped 1 previous similar message [ 2224.130060] Lustre: Skipped 1 previous similar message [ 2229.217744] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2229.231085] Lustre: Skipped 1 previous similar message [ 2234.338892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2234.350520] Lustre: Skipped 1 previous similar message [ 2236.384261] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2236.637131] Lustre: server umount lustre-MDT0000 complete [ 2240.738896] LustreError: 66443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779128553 with bad export cookie 10697599239658733318 [ 2240.750256] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2240.754059] LustreError: 66443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2241.254300] Lustre: server umount lustre-MDT0001 complete [ 2250.552774] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 14:22:42 (1779128562) [ 2262.431066] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2321.784808] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2338.099259] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2387.632094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2387.915321] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2387.933622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2387.990612] Lustre: lustre-MDT0000: new disk, initializing [ 2388.054603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2388.069687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2394.159581] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2407.661395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2407.818171] Lustre: 74066:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2407.895286] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2407.899355] Lustre: Skipped 1 previous similar message [ 2407.976798] Lustre: lustre-MDT0001: new disk, initializing [ 2408.101925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2408.157058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2408.180166] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2414.540623] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2423.297784] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2427.963086] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2437.771238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2438.251353] Lustre: lustre-OST0000: new disk, initializing [ 2438.255176] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2438.263377] Lustre: 75807:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2438.442526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2444.345082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2444.350250] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2444.479840] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2445.488806] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2455.688642] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2462.739479] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2463.008588] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2469.694618] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2469.941936] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2475.500011] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2475.503823] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.519352] LustreError: Skipped 1 previous similar message [ 2475.536343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2475.537704] Lustre: Skipped 1 previous similar message [ 2485.776186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2485.783060] Lustre: Skipped 3 previous similar messages [ 2487.265692] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2487.674945] Lustre: server umount lustre-OST0000 complete [ 2495.461690] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2495.465221] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2504.165042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2504.174775] Lustre: Skipped 6 previous similar messages [ 2507.232791] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2507.653558] Lustre: server umount lustre-MDT0000 complete [ 2509.285279] LustreError: 74078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2509.296423] LustreError: 74078:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2512.369301] LustreError: 74059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779128825 with bad export cookie 10697599239658734200 [ 2512.376243] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2512.380852] LustreError: 74059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2513.188300] Lustre: server umount lustre-MDT0001 complete [ 2525.942919] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 14:27:17 (1779128837) [ 2535.138682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2535.633927] LustreError: 78077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2535.751979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2541.028287] LustreError: 78078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2541.883767] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2545.157503] LustreError: 78077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2550.254961] LustreError: 78078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2553.993829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2560.218686] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2567.004336] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2571.794287] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2580.810806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2581.286970] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2581.290497] Lustre: Skipped 1 previous similar message [ 2589.177963] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2596.814858] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2601.767899] Lustre: 80411:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 8faaadc7-d485-44ca-a989-32f28d4c89d8 at adminstrative request [ 2605.032357] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2605.048335] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2615.212741] Lustre: MGS: Client 8faaadc7-d485-44ca-a989-32f28d4c89d8 (at 0@lo) reconnecting [ 2615.233364] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x947581d4855905e3 to 0x947581d485590883 [ 2615.251852] Lustre: MGC192.168.201.141@tcp: Connection restored to 0@lo (at 0@lo) [ 2642.473060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2642.721359] Lustre: lustre-OST0001: new disk, initializing [ 2642.728774] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2642.740327] Lustre: 80805:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2650.795293] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2652.910221] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2652.928241] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2653.145450] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2661.541560] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2668.523472] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2668.527020] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2668.532443] LustreError: Skipped 1 previous similar message [ 2668.579866] Lustre: Skipped 2 previous similar messages [ 2668.583823] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2668.589754] Lustre: Skipped 1 previous similar message [ 2671.640155] Lustre: server umount lustre-OST0001 complete [ 2673.660913] LustreError: 79675:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2673.680661] LustreError: 79675:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2677.732791] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2677.753578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2683.418049] Lustre: server umount lustre-OST0000 complete [ 2695.234689] Lustre: server umount lustre-MDT0000 complete [ 2698.216435] LustreError: 78072:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2698.236038] LustreError: 78072:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2699.787901] LustreError: 78058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779129012 with bad export cookie 10697599239658735747 [ 2699.795885] LustreError: 78058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2699.798905] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2700.435885] Lustre: server umount lustre-MDT0001 complete [ 2711.469308] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 14:30:23 (1779129023) [ 2713.135995] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2715.217781] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 14:30:26 (1779129026) [ 2717.744460] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2720.148673] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 14:30:31 (1779129031) [ 2722.472109] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2724.726840] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 14:30:36 (1779129036) [ 2726.260754] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2728.765242] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 14:30:39 (1779129039) [ 2730.914086] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2737.822624] loop: module loaded [ 2737.852588] loop0: detected capacity change from 0 to 4294967296 [ 2737.928517] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2739.738921] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2744.740686] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2748.877876] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2783.244862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2783.324798] LustreError: 84803:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2783.329899] LustreError: 84803:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2783.339928] LustreError: 84803:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2783.347961] LustreError: 84803:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2810.095697] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2824.106146] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2873.411033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2873.901272] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2873.973124] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2874.123258] Lustre: lustre-MDT0000: new disk, initializing [ 2874.281944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2874.292765] Lustre: Skipped 1 previous similar message [ 2874.325952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2879.512108] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2894.584641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2894.790707] Lustre: 87647:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2895.354418] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2902.659776] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2912.696812] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2916.570699] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2924.885761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2925.309732] Lustre: 89391:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2931.339645] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2931.546063] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2934.651265] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2944.772959] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2950.310517] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2950.607722] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2956.867817] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2957.228829] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2962.404788] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2962.434554] Lustre: Skipped 3 previous similar messages [ 2962.437857] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2962.440875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2962.452051] Lustre: Skipped 8 previous similar messages [ 2965.927507] Lustre: server umount lustre-OST0000 complete [ 2977.771173] LustreError: 88750:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2977.834529] Lustre: server umount lustre-MDT0000 complete [ 2982.420292] LustreError: 87639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779129295 with bad export cookie 10697599239658736370 [ 2982.427652] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2982.435114] LustreError: 87639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2983.093892] Lustre: server umount lustre-MDT0001 complete [ 2994.423767] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 14:35:05 (1779129305) [ 3004.206871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3004.867818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3004.873862] Lustre: Skipped 2 previous similar messages [ 3011.444805] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3022.131971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3028.348962] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3034.674482] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3039.494742] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3048.664327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3056.653022] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3063.230227] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3069.576366] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3069.867880] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3075.557973] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3075.857857] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3084.557146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3084.864601] Lustre: lustre-OST0001: new disk, initializing [ 3084.872819] Lustre: Skipped 2 previous similar messages [ 3084.882979] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3084.895106] Lustre: Skipped 3 previous similar messages [ 3084.904403] Lustre: 94661:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3092.623958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3092.634710] Lustre: Skipped 2 previous similar messages [ 3092.645867] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3092.819694] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3093.400023] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3102.638781] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3107.862639] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3108.132912] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3114.514485] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3114.756965] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3139.262796] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3170.437915] Lustre: Failing over lustre-MDT0000 [ 3170.813902] Lustre: server umount lustre-MDT0000 complete [ 3171.299694] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3171.302975] LustreError: Skipped 1 previous similar message [ 3171.309131] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3171.332925] Lustre: Skipped 3 previous similar messages [ 3171.351448] LustreError: 92372:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3171.379272] LustreError: 92372:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3190.241035] Lustre: 64643:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779129487/real 1779129487] req@00000000a6d674ff x1865551344599168/t0(0) o400->MGC192.168.201.141@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1779129503 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3190.273443] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3198.972846] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3198.978701] LDISKFS-fs (dm-0): recovery complete [ 3199.000827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3200.352338] LustreError: 97338:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3200.366384] LustreError: 97338:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@0000000037b4d98d x1865551344604800/t0(0) o250->MGC192.168.201.141@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1779129513 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3200.425588] LustreError: 97338:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3200.501666] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x947581d48559d2fe [ 3200.524186] Lustre: MGC192.168.201.141@tcp: Connection restored to 0@lo (at 0@lo) [ 3200.998920] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3201.730630] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3201.732961] Lustre: 91655:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3205.788326] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3206.116183] Lustre: 91661:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3206.131597] Lustre: 91661:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 3206.170054] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3206.278513] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3206.602143] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3206.949491] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3206.970361] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 3207.327112] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3207.335909] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 3207.354826] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3207.371862] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 3208.013813] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3208.037934] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 3208.385653] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3208.393588] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 3209.354120] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3209.368830] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 5 previous similar messages [ 3210.021403] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3210.037329] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 3210.688138] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3210.690538] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 3213.553792] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3213.590606] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 3214.236194] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3214.246626] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 3214.888554] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3214.902569] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 3221.611311] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3221.625346] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 24 previous similar messages [ 3222.284541] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3222.311266] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 3222.952641] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3222.960043] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 3237.646277] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3237.673522] Lustre: 97370:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 49 previous similar messages [ 3238.321652] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3238.341909] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 3238.992249] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3239.010291] LustreError: 97370:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 3261.056651] Lustre: 97370:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3261.077234] Lustre: 97370:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3261.113986] Lustre: 97370:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 9e1f4d52-eca7-46d5-9582-ae5e4017cfb6@192.168.201.41@tcp [ 3261.150762] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3261.161886] Lustre: 97370:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@0000000030b6bf05 x1865551326547584/t0(8589934767) o101->9e1f4d52-eca7-46d5-9582-ae5e4017cfb6@192.168.201.41@tcp:19/0 lens 592/0 e 0 to 0 dl 1779129584 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3261.212987] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3261.231356] Lustre: lustre-MDT0000: Denying connection for new client 9e1f4d52-eca7-46d5-9582-ae5e4017cfb6 (at 192.168.201.41@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3261.252209] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3261.331082] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3261.345239] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3261.349582] Lustre: Skipped 2 previous similar messages [ 3261.448478] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3261.448615] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 3287.525912] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3287.553120] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3287.584822] Lustre: Skipped 3 previous similar messages [ 3287.603744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3287.616705] Lustre: Skipped 2 previous similar messages [ 3291.584400] Lustre: server umount lustre-OST0000 complete [ 3303.180925] Lustre: server umount lustre-OST0001 complete [ 3314.719520] Lustre: server umount lustre-MDT0000 complete [ 3318.762037] LustreError: 91661:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3318.777281] LustreError: 91661:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 3319.863208] LustreError: 91642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779129632 with bad export cookie 10697599239658787582 [ 3319.868040] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3319.873521] LustreError: 91642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3320.194195] Lustre: server umount lustre-MDT0001 complete [ 3332.635744] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 14:40:43 (1779129643) [ 3340.847593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3340.857430] Lustre: *** cfs_fail_loc=197, val=0*** [ 3356.512649] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3361.632529] LustreError: 99329:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 3366.797896] Lustre: *** cfs_fail_loc=197, val=0*** [ 3366.805409] Lustre: Skipped 65 previous similar messages [ 3387.424268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3387.426263] Lustre: Skipped 4 previous similar messages [ 3395.543591] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3403.528550] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3406.727351] Lustre: server umount lustre-OST0000 complete [ 3419.463484] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 14:42:10 (1779129730) [ 3442.763265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3449.074157] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3457.729148] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3461.696147] LustreError: 100866:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3461.701878] LustreError: 100866:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3461.706490] LustreError: 100866:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3462.225858] Lustre: server umount lustre-MDT0000 complete [ 3502.321846] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3518.835032] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3564.875732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3565.281863] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3565.354939] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3565.515442] Lustre: lustre-MDT0000: new disk, initializing [ 3565.786822] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3571.593338] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3583.245750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.405080] Lustre: 104926:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3583.774472] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3588.566723] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3596.719384] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3600.380917] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3607.157902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3607.497571] Lustre: 106669:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3614.514684] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3614.517587] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3623.073898] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3628.349681] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3628.697360] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3634.021730] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3634.244031] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3640.295343] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3640.321568] LustreError: Skipped 3 previous similar messages [ 3640.330935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3640.352474] Lustre: Skipped 5 previous similar messages [ 3640.362476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3640.367228] Lustre: Skipped 9 previous similar messages [ 3651.040233] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3651.385927] Lustre: server umount lustre-OST0000 complete [ 3661.945199] Lustre: server umount lustre-MDT0000 complete [ 3665.893311] LustreError: 104935:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3665.903165] LustreError: 104935:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3666.799854] LustreError: 104920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779129979 with bad export cookie 10697599239658798131 [ 3666.803665] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3666.807021] LustreError: 104920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3667.122362] Lustre: server umount lustre-MDT0001 complete [ 3677.798829] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 14:46:28 (1779129988) [ 3687.622979] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 3690.797983] Key type lgssc unregistered [ 3691.140842] LNet: 108996:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3691.146225] LNetError: 108996:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3691.166335] LNet: Removed LNI 192.168.201.141@tcp [ 3692.114519] Key type .llcrypt unregistered [ 3692.119790] Key type ._llcrypt unregistered [ 3714.079662] Key type ._llcrypt registered [ 3714.081365] Key type .llcrypt registered [ 3714.228724] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3730.898901] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3732.840493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3732.866348] alg: No test for adler32 (adler32-zlib) [ 3734.105103] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 3734.334701] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 3736.089286] Key type lgssc registered [ 3737.572119] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3785.637930] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3785.667126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3787.142916] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3787.168593] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3787.234696] Lustre: lustre-MDT0000: new disk, initializing [ 3787.328531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3787.343968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3792.980286] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3805.586874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3806.038132] Lustre: lustre-OST0000: new disk, initializing [ 3806.046825] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3806.052063] Lustre: Skipped 1 previous similar message [ 3806.064829] Lustre: 113176:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3806.207931] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3814.019593] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3814.068697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3814.078463] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3814.110711] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3826.872245] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3827.109917] Lustre: lustre-OST0001: new disk, initializing [ 3827.118262] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3827.127892] Lustre: 114021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3827.233603] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3834.477892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3834.481813] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3834.513191] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3834.931877] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3849.702152] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3849.741130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3849.751629] Lustre: Skipped 1 previous similar message [ 3851.434453] Lustre: server umount lustre-MDT0000 complete [ 3860.221337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3866.604855] LustreError: 112380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779130179 with bad export cookie 6086586839283491473 [ 3866.633573] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3866.760128] Lustre: server umount lustre-OST0000 complete [ 3872.131613] Lustre: server umount lustre-OST0001 complete [ 3885.017541] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 14:49:56 (1779130196) [ 3897.809874] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3926.714904] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 14:50:38 (1779130238) [ 3973.144055] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3985.586060] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4029.865483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4030.249108] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4030.285798] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4030.361527] Lustre: lustre-MDT0000: new disk, initializing [ 4030.436739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4030.451733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4035.805748] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4048.058908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4048.209983] Lustre: 120729:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4048.289471] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4048.293163] Lustre: Skipped 1 previous similar message [ 4048.416921] Lustre: lustre-MDT0001: new disk, initializing [ 4048.567064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4048.621082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4048.640119] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4053.439843] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4062.316519] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4068.223452] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4077.755650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4078.475658] Lustre: lustre-OST0000: new disk, initializing [ 4078.489026] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4078.505949] Lustre: 122468:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4078.665693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4084.290547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4084.307849] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4084.489983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4086.263310] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4094.573309] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4100.086853] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4100.297215] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4105.728493] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4106.034809] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4110.310638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4110.312020] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4110.332594] LustreError: Skipped 1 previous similar message [ 4110.335346] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4110.389493] Lustre: Skipped 2 previous similar messages [ 4114.870117] Lustre: server umount lustre-OST0000 complete [ 4120.545855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4120.553129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4120.569400] Lustre: Skipped 1 previous similar message [ 4120.588827] Lustre: Skipped 2 previous similar messages [ 4125.666050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4125.680547] Lustre: Skipped 1 previous similar message [ 4126.079157] Lustre: server umount lustre-MDT0000 complete [ 4130.604635] LustreError: 120720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779130443 with bad export cookie 6086586839283492698 [ 4130.614313] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4130.614677] LustreError: 120720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 4131.045992] Lustre: server umount lustre-MDT0001 complete [ 4142.438838] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 14:54:13 (1779130453) [ 4151.487329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4152.255485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4157.410844] LustreError: 124734:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4157.444314] LustreError: 124734:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4157.831524] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4162.537678] LustreError: 124733:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4167.657128] LustreError: 124734:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4168.331559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4173.900912] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4180.682053] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4184.486900] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4192.448032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4192.918603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4192.922373] Lustre: Skipped 1 previous similar message [ 4200.240157] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4207.003704] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4211.983819] Lustre: *** cfs_fail_loc=159, val=0*** [ 4228.354255] Lustre: lustre-MDT0000: Client 7f90fd78-02a2-4ba0-8040-6cc605d7f4fe (at 192.168.201.41@tcp) reconnecting [ 4230.513257] Lustre: *** cfs_fail_loc=159, val=0*** [ 4230.515512] Lustre: Skipped 5 previous similar messages [ 4246.714970] Lustre: lustre-MDT0000: Client 7f90fd78-02a2-4ba0-8040-6cc605d7f4fe (at 192.168.201.41@tcp) reconnecting [ 4265.955535] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4265.961970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4265.970852] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4269.026818] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4270.697682] Lustre: server umount lustre-OST0000 complete [ 4276.207628] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4276.216840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4276.227099] Lustre: Skipped 1 previous similar message [ 4276.233838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4276.240000] Lustre: Skipped 2 previous similar messages [ 4282.158736] Lustre: server umount lustre-MDT0000 complete [ 4284.388972] LustreError: 126885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4284.444624] LustreError: 126885:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4286.396740] LustreError: 124715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779130599 with bad export cookie 6086586839283493580 [ 4286.404776] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4286.408275] LustreError: 124715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4286.782471] Lustre: server umount lustre-MDT0001 complete [ 4299.732247] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4303.384642] Key type lgssc unregistered [ 4303.823982] LNet: 128165:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4303.835800] LNetError: 128165:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4304.877357] LNet: Removed LNI 192.168.201.141@tcp [ 4306.223842] Key type .llcrypt unregistered [ 4306.226626] Key type ._llcrypt unregistered [ 4332.234256] Key type ._llcrypt registered [ 4332.241657] Key type .llcrypt registered [ 4332.397141] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4333.452244] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4333.528925] alg: No test for adler32 (adler32-zlib) [ 4334.735559] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 4334.967461] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4336.680194] Key type lgssc registered [ 4338.185679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4347.105139] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4347.121718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4348.692265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4353.704228] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4358.114537] LustreError: 129617:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4358.129190] LustreError: 129617:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4363.247182] LustreError: 129616:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4363.640677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4364.139618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4370.411741] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4377.677474] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4383.285084] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4391.832565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4392.356852] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4393.419844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4399.804581] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4406.919539] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4420.334123] Lustre: *** cfs_fail_loc=159, val=0*** [ 4436.707668] Lustre: lustre-MDT0000: Client 716cc5be-41e7-4395-bcd0-811fab2dde36 (at 192.168.201.41@tcp) reconnecting [ 4439.207257] Lustre: *** cfs_fail_loc=159, val=0*** [ 4455.726779] Lustre: lustre-MDT0001: Client 716cc5be-41e7-4395-bcd0-811fab2dde36 (at 192.168.201.41@tcp) reconnecting [ 4457.474348] Lustre: *** cfs_fail_loc=159, val=0*** [ 4457.479547] Lustre: Skipped 3 previous similar messages [ 4473.039701] Lustre: lustre-MDT0001: Client 716cc5be-41e7-4395-bcd0-811fab2dde36 (at 192.168.201.41@tcp) reconnecting [ 4479.784349] Lustre: 132317:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4483.686428] Lustre: *** cfs_fail_loc=159, val=0*** [ 4483.689257] Lustre: Skipped 4 previous similar messages [ 4499.215437] Lustre: lustre-MDT0000: Client c9d52848-ea4c-41cd-8075-ccc40196e7b4 (at 192.168.201.41@tcp) reconnecting [ 4501.627774] Lustre: *** cfs_fail_loc=159, val=0*** [ 4501.633789] Lustre: Skipped 14 previous similar messages [ 4517.151334] Lustre: lustre-MDT0000: Client c9d52848-ea4c-41cd-8075-ccc40196e7b4 (at 192.168.201.41@tcp) reconnecting [ 4519.117453] Lustre: 132611:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4526.561630] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4526.572192] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.609250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4527.586172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4527.595229] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4527.597928] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4531.937738] Lustre: server umount lustre-OST0000 complete [ 4537.838802] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4537.849319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4537.871522] Lustre: Skipped 1 previous similar message [ 4537.885704] Lustre: Skipped 2 previous similar messages [ 4542.638315] Lustre: server umount lustre-MDT0000 complete [ 4542.954303] LustreError: 132439:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4543.006938] LustreError: 132439:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4546.476939] LustreError: 129597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779130859 with bad export cookie 17922400864569563645 [ 4546.479695] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4546.486868] LustreError: 129597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4546.843643] Lustre: server umount lustre-MDT0001 complete [ 4568.963036] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4572.831968] Key type lgssc unregistered [ 4573.243112] LNet: 133685:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4573.253415] LNetError: 133685:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4574.311166] LNet: Removed LNI 192.168.201.141@tcp [ 4575.517526] Key type .llcrypt unregistered [ 4575.520928] Key type ._llcrypt unregistered [ 4604.753903] Key type ._llcrypt registered [ 4604.757339] Key type .llcrypt registered [ 4604.880874] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4607.341392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4607.372341] alg: No test for adler32 (adler32-zlib) [ 4608.260946] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 4608.396862] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4610.040458] Key type lgssc registered [ 4611.721613] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4624.588830] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4624.621352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4626.606113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4633.717647] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4636.132877] LustreError: 135138:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4636.180843] LustreError: 135138:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4641.255727] LustreError: 135137:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4644.406486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4644.969812] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4650.088833] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4657.203397] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4661.458958] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4670.149561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4670.571461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4671.638404] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4678.858793] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4686.418729] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4689.062041] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4696.485476] Lustre: 137382:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4701.688808] Lustre: 137431:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4703.922216] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4705.147402] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4710.534287] Lustre: 137578:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4711.519687] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4713.028868] Lustre: 137686:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4720.098205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4720.113627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4720.134513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4723.176742] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4723.177543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4723.211906] Lustre: Skipped 1 previous similar message [ 4724.662675] Lustre: server umount lustre-OST0000 complete [ 4731.873816] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4731.882141] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4731.899658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4738.529934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4738.540877] Lustre: Skipped 3 previous similar messages [ 4743.649736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4743.656437] Lustre: Skipped 1 previous similar message [ 4744.161075] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4744.446272] Lustre: server umount lustre-MDT0000 complete [ 4748.732600] LustreError: 135118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779131061 with bad export cookie 14835076339857765330 [ 4748.735716] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4748.745871] LustreError: 135118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4749.146139] Lustre: server umount lustre-MDT0001 complete [ 4760.698159] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4764.534899] Key type lgssc unregistered [ 4764.888696] LNet: 138722:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4764.907268] LNetError: 138722:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4764.937703] LNet: Removed LNI 192.168.201.141@tcp [ 4765.990173] Key type .llcrypt unregistered [ 4765.992943] Key type ._llcrypt unregistered [ 4793.171737] Key type ._llcrypt registered [ 4793.177910] Key type .llcrypt registered [ 4793.311879] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4794.916049] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4794.952720] alg: No test for adler32 (adler32-zlib) [ 4796.107671] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 4796.427017] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4798.144184] Key type lgssc registered [ 4799.667520] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4809.373161] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4809.427273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4810.994887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4816.514927] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4820.452290] LustreError: 140174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4820.471376] LustreError: 140174:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4825.572520] LustreError: 140175:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4827.017427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4827.530245] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4833.249993] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4839.979481] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4844.941513] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4852.970174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4853.633707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4855.695319] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4861.029436] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4868.341722] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4872.070791] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4878.137395] Lustre: *** cfs_fail_loc=159, val=0*** [ 4878.146710] Lustre: Skipped 1 previous similar message [ 4894.537413] Lustre: lustre-MDT0000: Client 549ebc67-3c19-4d32-bfab-dd51ee8ff4a6 (at 192.168.201.41@tcp) reconnecting [ 4901.857861] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4901.869520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4901.894126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4902.884398] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4902.900755] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4902.913049] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4907.798662] Lustre: server umount lustre-OST0000 complete [ 4913.122209] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4913.126625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4913.132543] Lustre: Skipped 1 previous similar message [ 4913.149996] Lustre: Skipped 2 previous similar messages [ 4918.080856] Lustre: server umount lustre-MDT0000 complete [ 4918.243624] LustreError: 140174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4918.268137] LustreError: 140174:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4922.086551] LustreError: 140155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779131235 with bad export cookie 3936823657486267254 [ 4922.093110] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4922.106959] LustreError: 140155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4922.614719] Lustre: server umount lustre-MDT0001 complete [ 4936.369641] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4939.956063] Key type lgssc unregistered [ 4940.366622] LNet: 143484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4940.374495] LNetError: 143484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4940.393167] LNet: Removed LNI 192.168.201.141@tcp [ 4941.569913] Key type .llcrypt unregistered [ 4941.572416] Key type ._llcrypt unregistered [ 4974.591765] Key type ._llcrypt registered [ 4974.593343] Key type .llcrypt registered [ 4974.811131] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4977.039867] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4977.057829] alg: No test for adler32 (adler32-zlib) [ 4978.406356] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 4978.708346] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4980.480247] Key type lgssc registered [ 4981.385940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4992.374073] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4992.427739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4994.116418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4999.018312] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5004.262420] LustreError: 144937:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5004.280958] LustreError: 144937:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5008.042097] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5008.599505] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5013.382913] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5020.124438] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5024.583563] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5032.262911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5032.765477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5033.787616] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5039.995256] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5040.397636] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5046.739760] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5067.484811] Lustre: lustre-MDT0001: evicting 88838beb-cff3-4dd1-b6b8-936ffedfbaef (at 192.168.201.41@tcp) by administrative request [ 5067.493302] Lustre: Skipped 1 previous similar message [ 5096.418945] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5096.424050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5096.451088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5096.463459] Lustre: Skipped 1 previous similar message [ 5101.539094] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5101.549905] Lustre: Skipped 1 previous similar message [ 5102.533191] Lustre: server umount lustre-OST0000 complete [ 5106.658854] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5106.659795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5106.676072] Lustre: Skipped 2 previous similar messages [ 5106.690340] Lustre: Skipped 1 previous similar message [ 5111.779125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5111.798293] Lustre: Skipped 1 previous similar message [ 5113.261771] Lustre: server umount lustre-MDT0000 complete [ 5116.898201] LustreError: 144938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5116.914789] LustreError: 144938:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5118.390395] LustreError: 144916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779131431 with bad export cookie 2219618591465112443 [ 5118.405170] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5118.818878] Lustre: server umount lustre-MDT0001 complete [ 5132.763246] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5135.952190] Key type lgssc unregistered [ 5136.260281] LNet: 148648:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5136.278246] LNetError: 148648:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5136.317359] LNet: Removed LNI 192.168.201.141@tcp [ 5137.346073] Key type .llcrypt unregistered [ 5137.353212] Key type ._llcrypt unregistered [ 5224.345178] Key type ._llcrypt registered [ 5224.347886] Key type .llcrypt registered [ 5224.484756] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5226.104240] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5226.207278] alg: No test for adler32 (adler32-zlib) [ 5227.274263] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 5227.507309] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5229.200248] Key type lgssc registered [ 5230.618571] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5240.650305] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5240.699466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5242.556888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5247.410748] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5252.073410] LustreError: 151176:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5252.105734] LustreError: 151176:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5257.195896] LustreError: 151175:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5258.115270] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5258.747970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5264.361356] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5271.945802] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5276.299856] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5284.555103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5284.897323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5288.001721] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5291.038853] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5297.753902] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5303.351903] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5311.979882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5311.988180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5312.006380] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5313.510746] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5313.515354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5313.524408] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5313.540604] Lustre: Skipped 1 previous similar message [ 5317.572331] Lustre: server umount lustre-OST0000 complete [ 5323.751361] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5323.773365] Lustre: Skipped 1 previous similar message [ 5323.777080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5323.790947] Lustre: Skipped 1 previous similar message [ 5328.118383] Lustre: server umount lustre-MDT0000 complete [ 5328.864828] LustreError: 151175:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5328.884097] LustreError: 151175:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5332.094737] LustreError: 151157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779131645 with bad export cookie 14321479042826059090 [ 5332.119985] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5332.700559] Lustre: server umount lustre-MDT0001 complete [ 5344.253688] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5346.945905] Key type lgssc unregistered [ 5347.199445] LNet: 154383:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5347.205156] LNetError: 154383:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5347.225691] LNet: Removed LNI 192.168.201.141@tcp [ 5347.926099] Key type .llcrypt unregistered [ 5347.927877] Key type ._llcrypt unregistered [ 5375.957419] Key type ._llcrypt registered [ 5375.965314] Key type .llcrypt registered [ 5376.095754] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5377.619135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5377.905627] alg: No test for adler32 (adler32-zlib) [ 5379.224908] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 5379.474415] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5381.250560] Key type lgssc registered [ 5382.797598] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5400.012276] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:15:11 (1779131711) [ 5421.651146] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 5431.964213] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5479.502042] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5479.516205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5480.944166] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5481.042496] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5481.226240] Lustre: lustre-MDT0000: new disk, initializing [ 5481.653567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5481.768368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5488.018919] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5500.007151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5500.140929] Lustre: 158662:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5500.204187] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5500.208195] Lustre: Skipped 1 previous similar message [ 5500.273680] Lustre: lustre-MDT0001: new disk, initializing [ 5500.489263] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5500.537191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5500.554663] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5505.897921] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5515.272204] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5520.058680] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5528.857666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5529.343305] Lustre: lustre-OST0000: new disk, initializing [ 5529.368036] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5529.381477] Lustre: 160403:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5529.519517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5535.920552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5535.929487] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5536.111100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5538.482817] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5547.138362] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5561.831494] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5561.834156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5561.839239] LustreError: Skipped 1 previous similar message [ 5561.841086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5561.867902] Lustre: Skipped 1 previous similar message [ 5561.884930] Lustre: Skipped 1 previous similar message [ 5563.731331] Lustre: server umount lustre-OST0000 complete [ 5572.067268] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5572.069106] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5572.092970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5575.012773] Lustre: server umount lustre-MDT0000 complete [ 5579.808176] LustreError: 158653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779131892 with bad export cookie 1809843840321887678 [ 5579.811090] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5579.816392] LustreError: 158653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5580.349380] Lustre: server umount lustre-MDT0001 complete [ 5595.138722] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5598.694114] Key type lgssc unregistered [ 5599.058504] LNet: 162179:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5599.068390] LNetError: 162179:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5599.094498] LNet: Removed LNI 192.168.201.141@tcp [ 5600.083091] Key type .llcrypt unregistered [ 5600.085664] Key type ._llcrypt unregistered [ 5635.289536] Key type ._llcrypt registered [ 5635.291567] Key type .llcrypt registered [ 5635.430212] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5637.330312] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5637.369569] alg: No test for adler32 (adler32-zlib) [ 5638.639933] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 5638.998230] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5640.824606] Key type lgssc registered [ 5642.477935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5653.719489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5653.732715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5655.220540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5660.536557] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5665.774297] LustreError: 163632:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5665.787735] LustreError: 163632:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5670.256111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5670.718433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5675.600947] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5682.007905] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5686.274818] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5693.889970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5694.250581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5700.413843] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5701.663204] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5708.350972] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5715.869307] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5715.887041] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5715.908357] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5715.918502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.929361] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5716.869518] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5716.883929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5716.898158] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5716.907390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.928566] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5717.971288] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5717.996887] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5718.012335] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5718.017296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.026994] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5719.117652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.372653] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5720.387421] Lustre: Skipped 1 previous similar message [ 5720.403577] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5720.409884] Lustre: Skipped 1 previous similar message [ 5720.412899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5720.416148] Lustre: Skipped 1 previous similar message [ 5720.418513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.428052] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5720.440449] Lustre: Skipped 1 previous similar message [ 5721.451218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.466027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.536277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.559814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5724.567964] Lustre: Skipped 3 previous similar messages [ 5724.584794] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5724.593249] Lustre: Skipped 3 previous similar messages [ 5724.608404] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5724.613733] Lustre: Skipped 3 previous similar messages [ 5724.634197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.650336] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5724.653288] Lustre: Skipped 3 previous similar messages [ 5725.738814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.243777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.199957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.328744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.430642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.592953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.671595] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5732.688173] Lustre: Skipped 6 previous similar messages [ 5732.715146] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5732.746243] Lustre: Skipped 6 previous similar messages [ 5732.757780] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5732.763230] Lustre: Skipped 6 previous similar messages [ 5732.773426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.790486] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5732.798220] Lustre: Skipped 6 previous similar messages [ 5733.772556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.704673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.863953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.931050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.800442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.649710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.458743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.588072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.617894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.533979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.645390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.862596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.165669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.281859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.170657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.208224] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5749.210019] Lustre: Skipped 15 previous similar messages [ 5749.223650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5749.230983] Lustre: Skipped 15 previous similar messages [ 5749.237672] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5749.243906] Lustre: Skipped 15 previous similar messages [ 5749.245537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.249321] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5749.252484] Lustre: Skipped 15 previous similar messages [ 5750.017073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.833710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.763971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.821647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.531919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.634649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.528030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.387340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.230749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.051732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.909869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.856339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.757074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.680173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.614870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.513637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.370305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.312371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.331332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.100503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.914816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.716638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.457230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.333182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.341959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.273227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.298217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.264181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.272309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.403037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.587955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.537385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.467974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.420912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.549406] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5781.556380] Lustre: Skipped 34 previous similar messages [ 5781.577664] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5781.586510] Lustre: Skipped 34 previous similar messages [ 5781.589851] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5781.595389] Lustre: Skipped 34 previous similar messages [ 5781.603253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.615173] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5781.623088] Lustre: Skipped 34 previous similar messages [ 5782.622246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.715112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.918421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.109558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.300170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.388244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.264044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.448533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.497993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.658576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.633795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.854610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.837978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.737882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.736975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.737663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.890494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.251079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.509708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.715964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.569627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.698154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.792127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.849372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.620648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.670074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.752752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.704744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.609068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.506501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.450364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.391326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.264714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.098499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.981665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.993399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.766259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.569489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.361612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.133686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.015728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.836302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.523158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.373220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.268430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.103411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.897937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.840740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.721563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.534459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.558703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.642517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.521633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.653234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.604740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.766914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.717569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.019261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.191760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.043629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.001490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.870916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.770296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.753601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.699407] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5845.710296] Lustre: Skipped 64 previous similar messages [ 5845.733867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5845.740829] Lustre: Skipped 64 previous similar messages [ 5845.743857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5845.751300] Lustre: Skipped 64 previous similar messages [ 5845.755225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.772978] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5845.776627] Lustre: Skipped 64 previous similar messages [ 5846.644787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.811367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.873213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.813035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.910463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.996823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.076902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.950248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.783855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.673264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.431229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.277521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.319647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.203714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.078291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.852751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.782313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.760821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.923814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.712887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.621798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.522611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.376647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.256691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.305803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.396980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.407788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.372625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.305472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.449392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.506481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.482927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.478739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.403464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.436848] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5879.447713] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5879.475926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.788540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.862082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.149218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.178420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.047247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.995212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.989578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.794252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.981669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.821508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.925083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.838568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.778745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.038802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.892897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.952611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.886631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.081666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.250184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.288608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.211120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.159775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.219832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.266702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.300879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.265429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.275505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.242340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.131949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.856539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.833850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.637310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.521575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.329373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.279620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.258929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.192203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.215476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.114637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.051763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.930239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.832917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.757031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.676547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.871651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.892026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.139090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.113511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.201995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.176755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.224342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.027336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.978590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.142738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.284358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.360814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.352523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.239307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.491057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.445081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.384280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.365092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.491853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.006735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.258276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.034386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.032179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.175345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.213965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.565045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.845151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.734516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.883139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.044593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.894587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.621220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.257767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.662503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.691508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.701918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.642858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.667552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.927864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.842361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.870059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.804786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.879654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.040865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.053269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.974993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.043884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.029250] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5974.039189] Lustre: Skipped 126 previous similar messages [ 5974.081805] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5974.099592] Lustre: Skipped 126 previous similar messages [ 5974.104977] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5974.115967] Lustre: Skipped 126 previous similar messages [ 5974.126949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.146433] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5974.149384] Lustre: Skipped 126 previous similar messages [ 5975.254811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.291411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.206721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.137523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.811936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.682080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.714594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.913530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.970288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.109617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.108471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.105399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.186620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.091862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.139895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.995326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.087873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.008761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.994893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.873758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.943656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.073102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.115866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.054541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.912058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.756295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.885064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.976311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.059618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.147900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.270674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.176848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.308131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.367840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.378738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.385889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.535496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.822069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.115858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.422502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.624196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.901433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.886588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.032838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.284071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.271102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.391315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.434731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.446405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.533681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.539674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.533149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.391747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.582729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.335239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.655576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.797099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.728055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.738032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.676156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.775926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.774410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.889774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.805388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.828732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.058375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.104531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.220490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.261070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.124566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.120220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.123914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.065600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.151117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.282209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.282753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.264217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.146285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.095772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.211619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.142643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.300081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.429455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.534282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.537611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.637481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.539911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.482554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.445513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.577769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.629350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.677212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.585431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.513419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.477570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.490973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.534563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.577656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.408053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.648356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.678970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.091087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.957236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.205765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.235765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.260456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.336229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.432104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.479694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.660691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.709866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.777321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.840584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.980886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.106283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.367479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.644175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.889538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.005183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.178730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.303577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.480676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.684915] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6103.695836] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6103.704946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.844994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.936366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.262437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.494979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.752889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.933853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.952394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.300170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.468146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.673777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.811513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.958840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.980272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.115399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.348491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.648972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.677727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.033409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.066829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.330968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.505544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.423129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.477641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.426914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.529445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.811390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.958264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.038975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.091088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.171674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.216501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.173855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.197863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.133783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.197407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.252164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.240323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.579200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.622690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.813068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.942530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.996375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.060054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.145881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.145499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.092769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.043193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.294733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.468765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.454199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.751953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.059217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.066384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.317705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.559911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.749382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.029995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.188356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.272960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.198779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.237792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.488889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.566163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.846621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.062912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.267726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.415447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.308482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.440565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.367299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.527173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.532828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.512751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.412743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.599884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.714267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.035238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.309279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.343323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.377148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.555251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.686100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.830713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.966926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.062188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.178782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.079779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.157041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.150225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.243083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.477094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.451078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.628216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.737071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.861141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.897889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.054103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.179796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.180565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.155370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.058113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.277765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.275356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.235554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.235875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.266213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.522289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.924778] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6223.939173] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6223.944946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.913723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.833487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.738194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.606349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.679787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.632981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.677312] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6230.683685] Lustre: Skipped 237 previous similar messages [ 6230.704655] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6230.719296] Lustre: Skipped 237 previous similar messages [ 6230.724392] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6230.731228] Lustre: Skipped 237 previous similar messages [ 6230.742452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.753073] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6230.763444] Lustre: Skipped 237 previous similar messages [ 6231.831748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.805378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.963579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.969232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.003794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.859227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.164657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.365789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.284275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.474306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.735264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.847246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.152767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.544957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.821816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.288481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.382954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.617717] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6251.631665] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6251.650448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.708927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.889161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.939662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.992166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.960827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.026793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.857986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.120587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.110597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.257283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.981254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.260883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.431711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.749697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.063610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.179048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.085798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.346804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.233409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.408623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.442047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.421499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.502757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.530834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.637586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.752224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.700474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.797229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.923514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.050762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.133059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.222706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.256118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.618814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.782318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.969096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.145109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.216578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.611142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.709089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.802762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.929225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.887812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.265119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.553388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.994456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.063519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.059428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.523336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.633881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.627929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.855586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.042314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.046389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.169338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.305324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.678668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.079694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.180671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.298091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.371169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.373577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.421335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.602073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.865976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.059594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.097714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.400378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.611054] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6329.627721] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6329.636882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.749300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.687928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.709434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.584618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.633368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.786742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.842193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.935563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.024640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.289177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.464223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.646883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.557885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.699273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.658105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.692205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.745215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.796166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.814627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.998837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.221254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.462905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.444840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.487862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.434300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.575369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.783971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.864634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.823553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.881834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.064611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.129378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.248189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.457892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.390074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.519393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.723950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.851776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.003664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.150641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.354621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.540128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.789125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.885748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.186152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.154409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.401083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.513778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.820526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.015094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.253642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.452769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.568230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.731992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.003991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.114164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.293555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.239525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.255732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.231844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.435376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.326449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.347244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.302539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.489673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.468544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.909273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.049164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.138198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.351232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.250286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.203590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.165916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.319773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.478934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.607642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.724769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.792751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.944492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.876060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.832653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.990076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.050340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.098688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.399450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.653361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.718179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.933880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.319722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.473902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.722424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.656387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.841993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.945210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.026920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.150058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.432713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.801204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.286697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.485586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.147454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.593563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.825173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.081671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.945124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.922839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.280736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.454798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.772745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.918275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.075194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.899760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.924734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.949433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.192248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.194529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.221450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.084112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.960109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.068671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.949934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.085791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.049214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.140253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.403487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.669117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.798554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.726656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.830809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.009723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.212629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.257543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.296438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.502872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.451810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.535767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.566842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.572633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.662628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.827196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.832530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.983578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.058685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.281305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.527984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.674036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.937800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.312248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.239587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.441764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.527177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.602606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.758920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.213560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.508107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.740401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.891820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.243743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.105227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.649790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.123724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.224725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.266330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.539404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.829499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.869789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.051697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.725822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.369110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.646925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.791735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.122334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.229977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.279573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.399763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.640101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.257376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.475844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.840489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.692762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.713598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.764930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.049763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.060460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.821573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.400875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.771682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.840900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.000081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.228226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.267851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.666946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.854067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.828847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.219534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.685831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.876801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.878360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.936664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.021699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.062631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.128486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.217035] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6561.222797] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6561.233362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.064801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.852291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.868896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.929298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.309298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.886933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.157971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.428381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.381857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.501976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.473049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.571414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.723183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.908888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.887749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.072617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.111984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.297292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.382061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.676718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.777055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.904064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.097221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.587790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.017383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.296658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.664585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.029732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.390027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.878827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.766574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.742815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.781830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.878199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.682639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.013724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.607536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.124894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.022758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.891625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.430360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.999349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.584356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.975346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.047429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.544364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.003292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.942752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.388254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.845148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.472806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.107897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.297836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.661330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.227832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.748236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.669634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.673920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.718173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.966740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.395628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.684783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.910881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.953516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.960037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.114370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.330625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.382384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.684067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.918352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.887459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.904957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.962699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.082234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.164288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.167742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.329660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.617973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.897983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.093629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.034648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.123424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.277094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.480885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.426245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.558807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.668875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.716515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.035045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.909479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.921024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.803371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.038615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.213538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.203971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.546598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.540835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.925746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.956533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.065629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.171862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.323517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.238913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.391179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.392749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.597345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.648166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.667339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.677640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.733149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.029418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.090126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.385504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.490610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.027530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.406669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.374201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.569240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.767769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.099886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.284978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.609154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.916151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.135862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.287059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.538924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.431934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.340704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.659819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.912246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.216246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.379885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.357205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.654560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.990868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.233952] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6724.248486] LustreError: 163652:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6724.278716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.253031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.574702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.766692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.002886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.366732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.695493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.137911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.250844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.515734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.753979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.019651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.352514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.783541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.016297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.110444] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6743.119040] Lustre: Skipped 440 previous similar messages [ 6743.145358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6743.155907] Lustre: Skipped 440 previous similar messages [ 6743.162221] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6743.169542] Lustre: Skipped 440 previous similar messages [ 6743.173939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.183326] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6743.186090] Lustre: Skipped 440 previous similar messages [ 6744.092718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.277681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.538170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.685577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.007072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.962776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.063347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.048453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.166644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.344781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.597539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.548951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.725272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.842879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.954309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.824221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.811125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.944714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.977179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.991991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.037439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.037470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.800823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.204775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.522605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.566741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.826391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.897055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.783180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.752660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.614549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.746148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.675416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.528068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.642266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.618772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.677798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.910378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.106682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.242395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.356935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.290859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.356644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.352695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.400483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.576846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.976647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.397238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.803855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.153913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.304288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.145389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.598745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.823757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.471909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.594870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.458758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.565844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.465903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.620892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.013174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.393640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.004585] LustreError: 175080:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7368.019978] LustreError: 163652:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8157.178111] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8157.180180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8157.181654] LustreError: Skipped 2 previous similar messages [ 8157.212386] Lustre: Skipped 62 previous similar messages [ 8157.231934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8158.177903] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8158.181302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8158.193505] LustreError: Skipped 5 previous similar messages [ 8158.212037] Lustre: Skipped 1 previous similar message [ 8161.210274] Lustre: server umount lustre-OST0000 complete [ 8168.420054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8168.432105] Lustre: Skipped 1 previous similar message [ 8172.266344] Lustre: server umount lustre-MDT0000 complete [ 8173.537662] LustreError: 165784:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8173.565153] LustreError: 165784:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8176.447459] LustreError: 210375:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779134489 with bad export cookie 13920422161644385761 [ 8176.451901] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8176.466308] LustreError: 210375:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8176.972738] Lustre: server umount lustre-MDT0001 complete [ 8189.112936] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8193.173641] Key type lgssc unregistered [ 8193.667699] LNet: 218606:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8193.673811] LNetError: 218606:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8193.706364] LNet: Removed LNI 192.168.201.141@tcp [ 8195.003757] Key type .llcrypt unregistered [ 8195.005283] Key type ._llcrypt unregistered [ 8225.060516] Key type ._llcrypt registered [ 8225.063877] Key type .llcrypt registered [ 8225.175091] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8226.814805] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8226.866854] alg: No test for adler32 (adler32-zlib) [ 8227.968228] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 8228.303585] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8229.952634] Key type lgssc registered [ 8231.179837] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8240.437939] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8240.471300] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8242.155755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8247.614551] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8252.385903] LustreError: 220059:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8252.402295] LustreError: 220059:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8256.050720] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8256.465217] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8261.661288] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8267.343155] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8271.826654] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8280.194202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8280.572512] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8287.262656] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8287.739658] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8293.579960] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8300.539679] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8303.509713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8303.532221] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8303.543422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8303.566850] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8303.568487] LustreError: 220079:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8307.178349] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8307.200644] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8307.236176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8308.202501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8308.228413] Lustre: Skipped 1 previous similar message [ 8311.545285] Lustre: lustre-OST0000: Not available for connect from 192.168.201.41@tcp (stopping) [ 8313.358272] LustreError: 221656:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8313.582386] Lustre: server umount lustre-OST0000 complete [ 8323.672173] LustreError: 220079:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8349.920769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8350.262622] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8356.782711] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8363.509208] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8363.560747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8363.576334] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8365.486582] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8368.678715] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8368.714910] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8370.138746] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff93d004952000.ost_server_uuid 50 [ 8371.871687] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff93d004952000.ost_server_uuid in FULL state after 0 sec [ 8378.848974] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8378.854500] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8378.858096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8378.864022] Lustre: Skipped 2 previous similar messages [ 8382.992601] Lustre: server umount lustre-OST0000 complete [ 8389.102285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8389.106026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8389.141111] Lustre: Skipped 1 previous similar message [ 8389.176368] Lustre: Skipped 2 previous similar messages [ 8393.765561] Lustre: server umount lustre-MDT0000 complete [ 8394.209752] LustreError: 220056:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8394.238100] LustreError: 220056:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8398.271809] LustreError: 220777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779134711 with bad export cookie 1094003712181448559 [ 8398.273578] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8398.280700] LustreError: 220777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8398.779628] Lustre: server umount lustre-MDT0001 complete [ 8412.370677] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8415.895809] Key type lgssc unregistered [ 8416.254241] LNet: 224704:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8416.272476] LNetError: 224704:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8416.302950] LNet: Removed LNI 192.168.201.141@tcp [ 8417.262363] Key type .llcrypt unregistered [ 8417.265484] Key type ._llcrypt unregistered [ 8460.711976] Key type ._llcrypt registered [ 8460.721574] Key type .llcrypt registered [ 8460.984779] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8463.405676] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8463.462279] alg: No test for adler32 (adler32-zlib) [ 8464.820977] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 8465.142263] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8466.880202] Key type lgssc registered [ 8468.960630] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8480.373041] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8480.396359] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8482.025511] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8487.211979] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8492.514978] LustreError: 226472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8492.532787] LustreError: 226472:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8496.015405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8496.461543] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8501.263958] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8504.820949] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8512.063988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8512.564030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8519.953599] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8527.934123] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8531.443184] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8531.686742] Lustre: lustre-OST0001: new disk, initializing [ 8531.694839] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8531.710916] Lustre: 228730:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8531.951511] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8537.289265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8537.301658] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8537.459798] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8539.027808] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8551.346079] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8559.608835] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8578.576206] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8578.581209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8578.581371] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8578.599112] Lustre: Skipped 2 previous similar messages [ 8578.618321] Lustre: Skipped 3 previous similar messages [ 8581.818632] Lustre: server umount lustre-MDT0000 complete [ 8583.664096] LustreError: 226473:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8583.698947] LustreError: 226473:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8586.538611] LustreError: 226452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779134899 with bad export cookie 6095250931941646832 [ 8586.543320] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8586.553554] LustreError: 226452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8586.863900] Lustre: server umount lustre-MDT0001 complete [ 8600.328081] Lustre: server umount lustre-OST0000 complete [ 8604.129362] Lustre: 225700:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1779134901/real 1779134901] req@ffff8972234c5c00 x1865558029070208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1779134917 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8604.173655] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8604.189940] Lustre: Skipped 1 previous similar message [ 8605.033812] Lustre: server umount lustre-OST0001 complete [ 8630.001431] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8642.049642] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8642.382815] LustreError: 231817:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8642.467031] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8647.655421] LustreError: 231818:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8648.120513] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8652.798637] LustreError: 231817:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8660.424313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8660.763583] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8666.080548] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8669.821461] Lustre: 232924:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8677.372379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8677.957920] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8685.167493] LustreError: 233277:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8685.196621] LustreError: 233277:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8685.644102] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8687.222582] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8694.855512] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8702.653964] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8711.762809] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8716.777873] Lustre: 234763:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8731.119195] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8731.135719] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8731.142099] Lustre: Skipped 2 previous similar messages [ 8731.162695] Lustre: Skipped 3 previous similar messages [ 8736.233261] LustreError: 233503:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8736.246453] LustreError: 233503:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 8736.314804] Lustre: server umount mylustre-MDT0000 complete [ 8741.487664] LustreError: 234765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779135054 with bad export cookie 6095250931941649639 [ 8741.489271] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8741.497660] LustreError: 234765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8742.216164] Lustre: server umount mylustre-MDT0001 complete [ 8756.886397] Lustre: server umount mylustre-OST0000 complete [ 8771.560387] Lustre: server umount mylustre-OST0001 complete [ 8802.029298] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8815.129877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8815.731481] LustreError: 237069:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8815.745359] LustreError: 237069:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 8815.811986] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8815.817165] Lustre: Skipped 1 previous similar message [ 8821.685959] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8833.019458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8839.402983] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8843.714166] Lustre: 238179:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8852.132089] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8860.930768] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8861.699275] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8872.811559] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8881.043473] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8889.461557] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8893.633387] Lustre: 240018:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8904.168620] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8904.173363] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8904.204324] Lustre: Skipped 2 previous similar messages [ 8904.218266] Lustre: Skipped 4 previous similar messages [ 8904.743129] Lustre: server umount tfs-MDT0000 complete [ 8909.288392] LustreError: 239840:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8909.329154] LustreError: 239840:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 8909.534960] LustreError: 237045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779135222 with bad export cookie 6095250931941651305 [ 8909.543961] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8909.555674] LustreError: 237045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8910.063576] Lustre: server umount tfs-MDT0001 complete [ 8924.245241] Lustre: server umount tfs-OST0000 complete [ 8939.609544] Lustre: server umount tfs-OST0001 complete [ 8967.023411] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8979.483902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8980.593093] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8980.606557] Lustre: Skipped 3 previous similar messages [ 8986.281118] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8997.977547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9003.983752] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9007.393731] Lustre: 243368:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9014.788883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9023.287106] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9025.401096] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 9033.288153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9041.156768] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9050.016052] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9054.321640] Lustre: 245206:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9063.639643] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:16:14 (1779135374) [ 9080.298306] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9080.309172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9080.333529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9080.340135] Lustre: Skipped 1 previous similar message [ 9084.906043] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9086.646258] Lustre: server umount lustre-MDT0000 complete [ 9090.025343] LustreError: 242258:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9090.068758] LustreError: 242258:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 9092.809612] LustreError: 242238:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779135405 with bad export cookie 6095250931941652747 [ 9092.814550] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9092.825788] LustreError: 242238:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9095.141920] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9095.162480] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9095.175473] Lustre: Skipped 4 previous similar messages [ 9095.191311] Lustre: Skipped 4 previous similar messages [ 9099.303513] Lustre: server umount lustre-MDT0001 complete [ 9121.248193] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9121.582346] Lustre: server umount lustre-OST0000 complete [ 9133.585916] Lustre: server umount lustre-OST0001 complete [ 9142.538805] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 9155.293767] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9202.403610] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 9202.403610] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9202.403610] [ 9202.443459] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9202.741291] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9202.781919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9202.890410] Lustre: lustre-MDT0000: new disk, initializing [ 9202.969691] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9202.978465] Lustre: Skipped 3 previous similar messages [ 9202.999501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9208.239114] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9221.833353] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 9221.833353] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9221.833353] [ 9221.882632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9222.004639] Lustre: 248372:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9222.050743] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9222.057112] Lustre: Skipped 1 previous similar message [ 9222.163832] Lustre: lustre-MDT0001: new disk, initializing [ 9222.321516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9222.328774] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9227.904496] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9240.985459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9241.244436] Lustre: lustre-OST0000: new disk, initializing [ 9241.247781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9241.252796] Lustre: 249921:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9242.444726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9242.475255] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9242.589398] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9249.591481] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9262.592453] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9262.824508] Lustre: lustre-OST0001: new disk, initializing [ 9262.828342] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9262.840411] Lustre: 250775:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9264.893979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9264.909139] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9265.014517] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9271.237025] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9285.488915] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:19:56 (1779135596) [ 9306.767743] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:20:17 (1779135617) [ 9326.576545] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9326.583809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9326.595047] Lustre: Skipped 1 previous similar message [ 9326.610103] Lustre: Skipped 3 previous similar messages [ 9331.395571] Lustre: server umount lustre-OST0000 complete [ 9339.878555] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9344.586690] Lustre: server umount lustre-MDT0000 complete [ 9347.063783] LustreError: 249230:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9347.105101] LustreError: 249230:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 9350.207706] LustreError: 248366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779135663 with bad export cookie 6095250931941654077 [ 9350.216661] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9350.219741] LustreError: 248366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9357.016212] Lustre: server umount lustre-MDT0001 complete [ 9371.096917] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9387.220670] Lustre: server umount lustre-OST0001 complete [ 9397.493153] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 9417.761815] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9471.998795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9472.258870] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9472.359682] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9472.429628] Lustre: lustre-MDT0000: new disk, initializing [ 9472.541642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9472.546224] Lustre: Skipped 3 previous similar messages [ 9472.572262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9479.033760] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9492.606399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9492.815262] Lustre: 255296:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9492.896770] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9492.901057] Lustre: Skipped 1 previous similar message [ 9492.959712] Lustre: lustre-MDT0001: new disk, initializing [ 9493.144023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9493.156962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9499.117132] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9508.894332] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9514.696661] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9523.202306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9523.590712] Lustre: lustre-OST0000: new disk, initializing [ 9523.599896] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9523.611269] Lustre: 257039:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9524.964904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9524.995248] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9525.223578] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9532.857467] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9543.093247] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9555.939696] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9555.942989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9555.968571] LustreError: Skipped 1 previous similar message [ 9555.969422] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9555.969426] Lustre: Skipped 6 previous similar messages [ 9556.002989] Lustre: Skipped 5 previous similar messages [ 9557.897278] Lustre: server umount lustre-OST0000 complete [ 9564.645882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9569.843732] Lustre: server umount lustre-MDT0000 complete [ 9576.150474] LustreError: 255288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779135889 with bad export cookie 6095250931941655547 [ 9576.151942] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9576.166777] LustreError: 255288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9576.956960] Lustre: server umount lustre-MDT0001 complete [ 9594.561709] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9597.894403] Key type lgssc unregistered [ 9598.343645] LNet: 258671:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9598.365875] LNetError: 258671:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9598.399194] LNet: Removed LNI 192.168.201.141@tcp [ 9599.593181] Key type .llcrypt unregistered [ 9599.596301] Key type ._llcrypt unregistered [ 9631.393441] Key type ._llcrypt registered [ 9631.398066] Key type .llcrypt registered [ 9631.544870] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9633.486418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9633.534813] alg: No test for adler32 (adler32-zlib) [ 9634.609631] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 9634.888519] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9636.680247] Key type lgssc registered [ 9638.338426] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9649.333268] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9649.357989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9651.500951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9657.805465] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9661.928112] LustreError: 260171:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9661.959287] LustreError: 260171:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9667.042655] LustreError: 260172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9668.908237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9669.798265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9675.828553] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9682.700503] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9688.891281] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9698.336677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9698.958649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9707.517724] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9714.925332] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9716.090849] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9719.778093] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9719.801744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9721.319269] Lustre: server umount lustre-OST0000 complete [ 9726.950981] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9726.967507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9726.978372] Lustre: Skipped 1 previous similar message [ 9727.000077] Lustre: Skipped 1 previous similar message [ 9732.069710] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9732.075773] Lustre: Skipped 1 previous similar message [ 9736.178193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9736.183899] Lustre: Skipped 1 previous similar message [ 9741.280227] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9741.281582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9741.308722] Lustre: Skipped 1 previous similar message [ 9741.662196] Lustre: server umount lustre-MDT0000 complete [ 9746.404970] LustreError: 260166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9746.426624] LustreError: 260166:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9746.702289] LustreError: 260153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779136059 with bad export cookie 14000082023427492460 [ 9746.711889] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9746.719981] LustreError: 260153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9747.193446] Lustre: server umount lustre-MDT0001 complete [ 9760.484137] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9765.088750] Key type lgssc unregistered [ 9765.678245] LNet: 263404:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9765.706654] LNetError: 263404:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9765.743163] LNet: Removed LNI 192.168.201.141@tcp [ 9767.159156] Key type .llcrypt unregistered [ 9767.173688] Key type ._llcrypt unregistered [ 9787.499328] Key type ._llcrypt registered [ 9787.501355] Key type .llcrypt registered [ 9787.604637] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9789.392797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9789.532390] alg: No test for adler32 (adler32-zlib) [ 9790.669329] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 9791.030015] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9792.778541] Key type lgssc registered [ 9794.113602] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9807.104916] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9807.145916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9822.816926] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9827.936250] LustreError: 264636:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 9854.689109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9862.542995] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9869.971532] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9873.574058] Lustre: server umount lustre-OST0000 complete [ 9896.989316] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9901.710160] Key type lgssc unregistered [ 9902.259930] LNet: 266138:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9902.265455] LNetError: 266138:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9902.283580] LNet: Removed LNI 192.168.201.141@tcp [ 9903.259831] Key type .llcrypt unregistered [ 9903.267126] Key type ._llcrypt unregistered [ 9953.754679] Key type ._llcrypt registered [ 9953.760743] Key type .llcrypt registered [ 9953.946989] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9955.828557] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9955.877133] alg: No test for adler32 (adler32-zlib) [ 9957.059627] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [ 9957.253156] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9959.032180] Key type lgssc registered [ 9960.212534] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9978.152552] loop0: detected capacity change from 0 to 204800000 [ 9978.439806] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9978.453461] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9978.857100] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9980.011799] loop0: detected capacity change from 0 to 204800000 [ 9980.160291] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9980.170366] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9980.668546] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9981.916445] loop0: detected capacity change from 0 to 204800000 [ 9982.127775] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9982.166510] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9982.364849] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9983.686978] loop0: detected capacity change from 0 to 204800000 [ 9984.006638] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9984.019477] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9984.204614] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9985.926224] loop0: detected capacity change from 0 to 204800000 [ 9985.993185] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9991.506573] loop0: detected capacity change from 0 to 204800000 [ 9991.534121] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9996.849520] loop0: detected capacity change from 0 to 204800000 [ 9996.896153] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10001.935865] loop0: detected capacity change from 0 to 204800000 [10001.979134] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10006.860996] loop0: detected capacity change from 0 to 204800000 [10006.939074] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10007.023287] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10007.144488] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10007.159570] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [10007.172544] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [10007.181291] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [10007.195642] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [10007.218861] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [10007.233856] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [10007.242427] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [10007.250362] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [10007.259209] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [10007.267445] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [10007.275473] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [10007.286379] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [10007.293575] Lustre: 269595:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [10013.923961] Lustre: server umount MGS complete [10015.279064] loop0: detected capacity change from 0 to 204800000 [10015.427557] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10016.303761] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10016.384935] Lustre: lustre-MDT0000: Aborting client recovery [10017.411943] loop1: detected capacity change from 0 to 204800000 [10017.540159] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10017.662873] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10017.681958] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [10017.696900] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [10017.711834] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [10017.728546] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [10017.744117] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [10017.756815] Lustre: 270028:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [10017.820118] Lustre: Found index 1 for lustre-MDT0001, updating log [10018.127463] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10018.249899] Lustre: lustre-MDT0001: Aborting client recovery [10019.506491] loop2: detected capacity change from 0 to 204800000 [10019.770743] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10019.897963] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10019.945759] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10019.977987] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [10019.994187] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [10020.006610] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10020.050220] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10020.075073] Lustre: 270107:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [10020.261787] Lustre: Found index 0 for lustre-OST0000, updating log [10020.335690] Lustre: 270125:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10020.485705] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10020.510784] Lustre: lustre-OST0000: Aborting recovery [10021.989373] loop3: detected capacity change from 0 to 204800000 [10022.306844] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10022.436086] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10022.464965] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10022.483702] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10022.505815] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [10022.536113] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [10022.566120] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [10022.589637] Lustre: 270190:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10022.607938] Lustre: Found index 1 for lustre-OST0001, updating log [10022.643155] Lustre: 270196:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10022.834397] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10022.889878] Lustre: lustre-OST0001: Aborting recovery [10026.932693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10026.985798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [10026.991561] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10027.000302] LustreError: 269972:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [10027.091212] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [10027.969525] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [10027.976170] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [10033.405632] LustreError: 269939:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 680us: rc = -2 [10033.914739] LustreError: 269939:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 750us: rc = -2 [10033.933254] LustreError: 269939:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [10048.494477] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10048.495756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10048.516521] Lustre: Skipped 2 previous similar messages [10048.540928] Lustre: Skipped 3 previous similar messages [10053.603142] LustreError: 269941:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10053.610254] LustreError: 269941:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10053.656562] Lustre: server umount lustre-MDT0000 complete [10054.667949] LustreError: 269923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779136367 with bad export cookie 7677563660474691164 [10054.713883] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10058.721480] LustreError: 269938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10058.723822] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10058.743344] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10058.747445] LustreError: 269938:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10058.787247] Lustre: Skipped 2 previous similar messages [10059.745373] LustreError: 270384:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10059.748476] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10059.785815] LustreError: 270384:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10059.791141] Lustre: Skipped 2 previous similar messages [10061.469598] Lustre: server umount lustre-MDT0001 complete [10068.934526] Lustre: server umount lustre-OST0000 complete [10076.642454] Lustre: server umount lustre-OST0001 complete [10086.045763] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:33:17 (1779136397) [10119.543668] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [10134.272048] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10182.277037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10182.686204] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10182.767401] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10182.866060] Lustre: lustre-MDT0000: new disk, initializing [10182.923423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10182.937703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10188.116635] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10199.843543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10200.066430] Lustre: 274241:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10200.144752] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10200.149814] Lustre: Skipped 1 previous similar message [10200.216988] Lustre: lustre-MDT0001: new disk, initializing [10200.310284] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10200.376068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10200.397961] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10205.521701] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10214.490867] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10218.426765] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10226.612939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10227.011911] Lustre: lustre-OST0000: new disk, initializing [10227.014846] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10227.019618] Lustre: 275985:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10227.118734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10234.730760] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10235.480800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10235.508675] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10235.727369] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10244.743674] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10251.046751] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10261.628866] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10264.836128] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10271.622285] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10317.793782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10317.799756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10317.808050] Lustre: Skipped 1 previous similar message [10317.841610] Lustre: Skipped 1 previous similar message [10319.302606] Lustre: server umount lustre-OST0000 complete [10323.427302] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10323.440734] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10323.481736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10328.038539] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10329.659130] Lustre: server umount lustre-MDT0000 complete [10333.156210] LustreError: 274254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10333.193961] LustreError: 274254:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10334.161110] LustreError: 277174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779136647 with bad export cookie 7677563660474693768 [10334.169683] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10334.178882] LustreError: 277174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [10334.804575] Lustre: server umount lustre-MDT0001 complete [10345.320766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10350.481969] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10355.855704] Lustre: server umount MGS complete [10372.043369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10372.958129] LustreError: 279572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10373.005551] LustreError: 279585:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10373.153588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10380.474825] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10383.332321] LustreError: 279573:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10383.383819] LustreError: 279573:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10390.677929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10396.877326] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10403.470932] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10407.630582] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10415.973731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10416.114130] LustreError: 281169:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10416.136367] LustreError: 281169:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10422.983862] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10429.122529] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10469.861424] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10469.866358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10469.891768] Lustre: Skipped 1 previous similar message [10469.913057] Lustre: Skipped 3 previous similar messages [10471.547148] Lustre: server umount lustre-OST0000 complete [10478.572111] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10490.336955] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10490.339577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10490.365452] Lustre: Skipped 6 previous similar messages [10490.894741] Lustre: server umount lustre-MDT0000 complete [10495.471812] LustreError: 279573:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10495.500471] LustreError: 279573:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10495.568234] LustreError: 279552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779136808 with bad export cookie 7677563660474695553 [10495.570887] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10495.576555] LustreError: 279552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10496.331930] Lustre: server umount lustre-MDT0001 complete [10509.565835] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [10513.754451] Key type lgssc unregistered [10514.158837] LNet: 283168:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10514.171307] LNetError: 283168:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10514.196825] LNet: Removed LNI 192.168.201.141@tcp [10515.373101] Key type .llcrypt unregistered [10515.379364] Key type ._llcrypt unregistered [10566.354787] Key type ._llcrypt registered [10566.358787] Key type .llcrypt registered [10566.481849] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [10584.124925] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10586.056270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10586.132566] alg: No test for adler32 (adler32-zlib) [10587.355805] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [10587.712108] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [10589.537313] Key type lgssc registered [10590.951848] Lustre: Echo OBD driver; http://www.lustre.org/ [10635.935625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10635.986739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10637.513750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10637.548967] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10637.643982] Lustre: lustre-MDT0000: new disk, initializing [10637.744148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10637.783744] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10644.697596] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10659.893323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10660.054787] Lustre: 287152:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10660.122466] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10660.125722] Lustre: Skipped 1 previous similar message [10660.220525] Lustre: lustre-MDT0001: new disk, initializing [10660.366886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10660.461165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10660.480288] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10666.331327] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10676.084575] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10680.709315] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10690.035082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10690.342878] Lustre: lustre-OST0000: new disk, initializing [10690.345821] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10690.357387] Lustre: 288895:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10690.486952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10697.788906] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10698.870683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10698.892547] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10699.048763] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10707.361924] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10715.895534] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10725.106332] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10736.495726] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10744.371556] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10786.275187] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10786.278576] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10786.314366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10786.323366] Lustre: Skipped 1 previous similar message [10786.367423] Lustre: Skipped 1 previous similar message [10791.396144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10791.407242] Lustre: Skipped 1 previous similar message [10796.528041] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10796.539022] Lustre: Skipped 1 previous similar message [10797.024151] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10797.242089] Lustre: server umount lustre-OST0000 complete [10801.634208] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10801.646912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10801.666137] Lustre: Skipped 1 previous similar message [10801.687103] Lustre: Skipped 1 previous similar message [10806.758401] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10806.763825] Lustre: Skipped 1 previous similar message [10807.346685] Lustre: server umount lustre-MDT0000 complete [10811.516187] LustreError: 289639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779137124 with bad export cookie 17495096738211321011 [10811.518812] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10811.528470] LustreError: 289639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10811.863729] Lustre: server umount lustre-MDT0001 complete [10821.266949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10826.440670] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10832.404607] Lustre: server umount MGS complete [10846.557559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10846.965568] LustreError: 292700:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10847.077158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10852.327849] LustreError: 292687:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10852.385811] LustreError: 292687:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10853.110513] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10857.467455] LustreError: 292688:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10862.562930] LustreError: 292687:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10864.576336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10864.910256] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10870.581864] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10877.180276] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10881.773897] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10890.698132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10891.260715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10899.289199] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10905.692472] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10938.341852] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10938.351138] LustreError: Skipped 1 previous similar message [10938.352814] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10938.375658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10940.397172] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10940.397260] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10944.494604] Lustre: server umount lustre-OST0000 complete [10950.627260] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10950.650593] Lustre: Skipped 1 previous similar message [10955.749214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10955.761643] Lustre: Skipped 5 previous similar messages [10956.173708] Lustre: server umount lustre-MDT0000 complete [10960.744201] LustreError: 294305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779137273 with bad export cookie 17495096738211322950 [10960.756463] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10961.465693] Lustre: server umount lustre-MDT0001 complete [10976.037355] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [10979.743469] Key type lgssc unregistered [10980.213753] LNet: 296278:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10980.221391] LNetError: 296278:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10980.245297] LNet: Removed LNI 192.168.201.141@tcp [10981.699078] Key type .llcrypt unregistered [10981.710367] Key type ._llcrypt unregistered [11008.523759] Key type ._llcrypt registered [11008.527655] Key type .llcrypt registered [11008.681956] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [11024.559557] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [11025.857703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11025.919600] alg: No test for adler32 (adler32-zlib) [11027.213169] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [11027.514901] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [11029.264212] Key type lgssc registered [11030.763174] Lustre: Echo OBD driver; http://www.lustre.org/ [11058.867745] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [11077.774528] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11077.839739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11079.122644] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11079.155794] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11079.239982] Lustre: lustre-MDT0000: new disk, initializing [11079.317566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11079.332574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11083.759685] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [11096.393506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11096.833202] Lustre: lustre-OST0000: new disk, initializing [11096.841877] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11096.852358] Lustre: Skipped 1 previous similar message [11096.862151] Lustre: 300327:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11096.962616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11102.924760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11102.938824] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11102.975597] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11104.684525] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12070.900979] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12070.918979] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12071.995686] Lustre: server umount lustre-MDT0000 complete [12076.722398] LustreError: 299529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779138389 with bad export cookie 18146029749026451831 [12076.736982] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12076.741428] LustreError: 299529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12077.021854] Lustre: server umount lustre-OST0000 complete [12110.838405] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [12123.514767] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12172.587465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12172.943298] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12172.973100] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12173.032477] Lustre: lustre-MDT0000: new disk, initializing [12173.164631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12173.200786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12177.298285] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12189.959255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12190.167200] Lustre: 304654:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12190.239568] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12190.249964] Lustre: Skipped 1 previous similar message [12190.398701] Lustre: lustre-MDT0001: new disk, initializing [12190.539497] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12190.580781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12190.600595] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12196.147861] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12205.105180] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12209.871831] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12217.710828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12218.144414] Lustre: lustre-OST0000: new disk, initializing [12218.149269] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12218.159111] Lustre: 306392:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12218.289559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12224.649372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12224.668320] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12224.844506] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12225.169175] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12233.981862] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12239.110882] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12239.327597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12245.039453] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12245.402865] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12250.608541] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12250.612164] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12250.616712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12250.637734] Lustre: Skipped 1 previous similar message [12250.644546] LustreError: Skipped 1 previous similar message [12253.881484] Lustre: server umount lustre-OST0000 complete [12260.835130] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12260.860934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12260.864544] Lustre: Skipped 1 previous similar message [12260.884280] Lustre: Skipped 2 previous similar messages [12264.657703] Lustre: server umount lustre-MDT0000 complete [12269.172176] LustreError: 304644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779138582 with bad export cookie 18146029749029392650 [12269.189315] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12269.727838] Lustre: server umount lustre-MDT0001 complete [12274.066526] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12275.813550] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:09:47 (1779138587) [12283.129714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12283.597556] LustreError: 308559:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12283.625783] LustreError: 308559:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12283.789419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12288.402230] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12289.002773] LustreError: 308560:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12293.089885] LustreError: 308559:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12297.562690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12297.940462] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12303.764184] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12310.101977] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12314.112587] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12321.938974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12322.255135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12329.794914] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12336.796933] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12345.581741] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12345.948583] Lustre: lustre-OST0001: new disk, initializing [12345.957306] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12345.969682] Lustre: 311054:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12346.121341] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12352.104242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12352.111698] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12352.207381] LustreError: 311329:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12353.254315] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12353.265677] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12353.340902] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12354.280915] LustreError: 311329:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12356.333179] LustreError: 311329:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12356.341959] LustreError: 311329:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12360.492552] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12360.534740] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [12363.907655] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12364.195529] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12368.730898] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff93d0118e7000.ost_server_uuid 50 [12368.739952] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12368.739964] LustreError: 311337:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [12370.289675] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff93d0118e7000.ost_server_uuid in FULL state after 0 sec [12374.700426] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff93d0118e7000.ost_server_uuid 50 [12376.465773] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff93d0118e7000.ost_server_uuid in FULL state after 0 sec [12378.057918] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12393.442187] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12393.446486] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12393.471622] LustreError: Skipped 1 previous similar message [12393.472203] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12393.525612] Lustre: Skipped 1 previous similar message [12398.574116] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12398.582683] Lustre: Skipped 3 previous similar messages [12406.240302] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12406.363940] Lustre: server umount lustre-OST0001 complete [12408.826991] LustreError: 310154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12408.866132] LustreError: 310154:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12413.929144] LustreError: 310154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12413.944760] LustreError: 310154:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12415.458021] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12415.460564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12415.466140] LustreError: Skipped 1 previous similar message [12415.467930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12415.477041] Lustre: Skipped 1 previous similar message [12415.507334] Lustre: Skipped 5 previous similar messages [12421.121031] Lustre: server umount lustre-OST0000 complete [12426.209272] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12426.225281] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12431.942715] Lustre: server umount lustre-MDT0000 complete [12434.405355] LustreError: 308553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12434.447516] LustreError: 308553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12436.309517] LustreError: 308539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779138749 with bad export cookie 18146029749029393532 [12436.312748] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12436.320307] LustreError: 308539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12436.899445] Lustre: server umount lustre-MDT0001 complete [12449.746493] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [12453.370366] Key type lgssc unregistered [12453.748081] LNet: 313503:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12453.757802] LNetError: 313503:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12453.772914] LNet: Removed LNI 192.168.201.141@tcp [12455.108307] Key type .llcrypt unregistered [12455.110069] Key type ._llcrypt unregistered [12483.072225] Key type ._llcrypt registered [12483.073990] Key type .llcrypt registered [12483.175811] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12484.273914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12484.343220] alg: No test for adler32 (adler32-zlib) [12485.697991] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [12485.924446] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [12487.568196] Key type lgssc registered [12488.901254] Lustre: Echo OBD driver; http://www.lustre.org/ [12498.467989] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12498.493027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12500.152402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12506.094895] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12510.699298] LustreError: 314969:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12510.723224] LustreError: 314969:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12514.249863] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12515.830861] LustreError: 314968:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12520.941381] LustreError: 314969:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12525.032535] LustreError: 314968:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12525.346868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12525.927101] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12532.496733] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12544.997119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12545.391315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12546.410964] LustreError: 316520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12546.419458] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12553.867184] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12555.757859] LustreError: 316520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12555.767789] LustreError: 316520:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12563.264114] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12571.989049] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12572.366150] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12577.804451] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12579.664568] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12586.160372] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [12594.348542] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12594.769741] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12703.733996] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12703.769897] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12703.796965] Lustre: Skipped 1 previous similar message [12707.853736] Lustre: server umount lustre-OST0001 complete [12708.855621] LustreError: 316519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12708.883070] LustreError: 316519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [12719.074341] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12719.109147] Lustre: Skipped 1 previous similar message [12719.121798] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12721.899938] Lustre: server umount lustre-OST0000 complete [12729.313885] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12729.338692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12729.354964] Lustre: Skipped 2 previous similar messages [12729.369973] Lustre: Skipped 1 previous similar message [12733.030387] Lustre: server umount lustre-MDT0000 complete [12737.848194] LustreError: 318281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779139050 with bad export cookie 295045598296808708 [12737.848985] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12737.870402] LustreError: 318281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12738.750331] Lustre: server umount lustre-MDT0001 complete [12753.028209] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [12755.684550] Key type lgssc unregistered [12756.046317] LNet: 319524:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12756.063084] LNetError: 319524:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12756.090906] LNet: Removed LNI 192.168.201.141@tcp [12757.014099] Key type .llcrypt unregistered [12757.023143] Key type ._llcrypt unregistered [12810.345596] Key type ._llcrypt registered [12810.347688] Key type .llcrypt registered [12810.446032] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12812.220960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12812.268406] alg: No test for adler32 (adler32-zlib) [12813.453925] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [12813.779607] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [12815.489324] Key type lgssc registered [12817.278150] Lustre: Echo OBD driver; http://www.lustre.org/ [12829.513474] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12829.527313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12830.886341] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12836.290306] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12840.419860] LustreError: 321856:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12840.430464] LustreError: 321856:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12845.543465] LustreError: 321855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12846.995616] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12847.435108] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12851.744967] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12855.279100] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12861.733896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12862.189172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12869.700392] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12872.442138] LustreError: 323309:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12875.501505] LustreError: 323309:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12875.520647] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12875.524528] LustreError: 323309:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12880.333936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12880.437379] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12885.502095] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12887.064281] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [12898.025882] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12905.771655] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12919.271114] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12919.285940] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12919.298827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12921.315153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12921.322386] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12921.334995] Lustre: Skipped 2 previous similar messages [12921.369135] Lustre: Skipped 2 previous similar messages [12925.005914] Lustre: server umount lustre-MDT0000 complete [12926.441360] LustreError: 321855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12926.484144] LustreError: 321855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12929.176911] LustreError: 321835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779139242 with bad export cookie 2567843697795914052 [12929.182548] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12929.185255] LustreError: 321835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12929.448808] Lustre: server umount lustre-MDT0001 complete [12943.014699] Lustre: server umount lustre-OST0000 complete [12956.218022] Lustre: server umount lustre-OST0001 complete [12976.005241] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [12988.404141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12988.936093] LustreError: 327043:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12989.048246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12994.179167] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13003.693796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13004.101112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13008.735533] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13011.830239] Lustre: 328149:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [13019.553616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13027.009870] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13029.171253] LustreError: 328502:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13029.207759] LustreError: 328502:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [13034.294714] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [13036.613342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13036.919316] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13036.921638] Lustre: Skipped 1 previous similar message [13042.199400] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [13043.828662] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13052.939894] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13057.772206] Lustre: 329988:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [13065.698073] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13065.707357] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13065.742832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13065.756485] Lustre: Skipped 1 previous similar message [13067.749288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13067.749889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13067.760690] Lustre: Skipped 2 previous similar messages [13067.782126] Lustre: Skipped 2 previous similar messages [13070.490988] Lustre: server umount lustre-MDT0000 complete [13072.878678] LustreError: 328783:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13072.899421] LustreError: 328783:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [13074.902293] LustreError: 327023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779139387 with bad export cookie 2567843697795915753 [13074.904091] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13074.910502] LustreError: 327023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13075.297111] Lustre: server umount lustre-MDT0001 complete [13089.826741] Lustre: server umount lustre-OST0000 complete [13104.792838] Lustre: server umount lustre-OST0001 complete [13109.400205] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [13111.487965] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [13113.545658] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:23:45 (1779139425) [13140.082510] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [13151.256722] loop0: detected capacity change from 0 to 1099511627776 [13151.726484] SGI XFS with ACLs, security attributes, quota, no debug enabled [13151.738686] *** VALIDATE xfs *** [13151.752534] XFS (loop0): Mounting V5 Filesystem [13151.784766] XFS (loop0): Ending clean mount [13153.813371] loop1: detected capacity change from 0 to 18691697672192 [13283.991182] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [13295.025682] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [13340.312287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13340.675397] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13340.718748] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13340.878374] Lustre: lustre-MDT0000: new disk, initializing [13340.992194] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13341.018957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13345.969975] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13358.252778] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13358.396398] Lustre: 335097:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13358.454241] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13358.459170] Lustre: Skipped 1 previous similar message [13358.546303] Lustre: lustre-MDT0001: new disk, initializing [13358.839563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13358.855204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13364.644398] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13373.687163] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13377.607509] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13385.134075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13385.346970] Lustre: lustre-OST0000: new disk, initializing [13385.350954] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13385.355643] Lustre: 336838:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13386.510769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13386.533889] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13386.694592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13392.977424] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13401.347184] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13407.085107] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13407.365574] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13412.409912] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13412.623105] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13417.447158] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13417.452666] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13417.454856] LustreError: Skipped 1 previous similar message [13417.455899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13417.455906] Lustre: Skipped 1 previous similar message [13417.494382] Lustre: Skipped 1 previous similar message [13427.682267] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13427.692154] Lustre: Skipped 3 previous similar messages [13429.216117] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13429.418545] Lustre: server umount lustre-OST0000 complete [13435.362300] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13435.372562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13440.518748] Lustre: server umount lustre-MDT0000 complete [13443.044833] LustreError: 335108:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13443.075439] LustreError: 335108:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [13444.721436] LustreError: 335090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779139757 with bad export cookie 2567843697795917300 [13444.733219] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13445.218680] Lustre: server umount lustre-MDT0001 complete [13455.892400] XFS (loop0): Unmounting Filesystem [13462.120206] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:29:32 (1779139772) [13471.338729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13472.018678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13472.026297] Lustre: Skipped 2 previous similar messages [13478.694204] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13488.654143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13494.351964] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13500.304934] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13504.123512] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13512.468219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13520.963530] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13527.421962] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13538.785697] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13538.792547] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13538.796164] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13538.801993] Lustre: Skipped 2 previous similar messages [13538.812862] Lustre: Skipped 5 previous similar messages [13542.577486] Lustre: server umount lustre-OST0000 complete [13562.336141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13562.792278] Lustre: server umount lustre-MDT0000 complete [13566.280663] LustreError: 339234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779139879 with bad export cookie 2567843697795918189 [13566.290143] LustreError: 339234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13566.298924] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13566.946543] Lustre: server umount lustre-MDT0001 complete [13580.133909] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [13583.427471] Key type lgssc unregistered [13583.904926] LNet: 342529:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13583.911559] LNetError: 342529:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13584.939059] LNet: Removed LNI 192.168.201.141@tcp [13586.561161] Key type .llcrypt unregistered [13586.563689] Key type ._llcrypt unregistered [13621.286628] Key type ._llcrypt registered [13621.289634] Key type .llcrypt registered [13621.462768] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [13623.256686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13623.340742] alg: No test for adler32 (adler32-zlib) [13624.573529] Lustre: Lustre: Build Version: 2.17.53_24_g2ff46d7 [13624.920864] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [13626.656394] Key type lgssc registered [13628.257707] Lustre: Echo OBD driver; http://www.lustre.org/ [13638.933406] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13638.944843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13640.450594] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13645.595876] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13650.921238] LustreError: 344136:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13650.941535] LustreError: 344136:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13655.825685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13656.034339] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [13656.186834] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13661.288856] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13668.346643] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13672.465779] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13681.092483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13681.543561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13689.329227] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13696.730583] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13710.051650] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [13711.332084] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13711.334436] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13711.341092] Lustre: Skipped 1 previous similar message [13711.356800] Lustre: Skipped 1 previous similar message [13714.987865] Lustre: server umount lustre-MDT0001 complete [13715.138130] LustreError: 344132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13716.449851] LustreError: 344847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13716.467969] LustreError: 344847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [13720.253858] LustreError: 344130:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13725.402335] LustreError: 344130:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13725.413694] LustreError: 344130:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13726.902492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13727.088419] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13727.098963] Lustre: Found index 1 for lustre-MDT0001, updating log [13727.149440] Lustre: 344123:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13727.514513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13731.811825] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [13732.869749] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13736.971394] LustreError: 347381:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13736.981493] LustreError: 347381:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13736.993787] LustreError: 344126:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13737.117686] LustreError: 347384:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [13737.142364] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13737.142364] [14037.754451] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [14038.003167] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14038.039793] Lustre: Skipped 1 previous similar message [14042.833638] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [14042.851238] Lustre: Skipped 2 previous similar messages [14044.422878] Lustre: server umount lustre-MDT0001 complete [14047.942923] LustreError: 344130:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [14047.965868] LustreError: 344130:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [14056.668340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14056.841432] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14056.845406] Lustre: Found index 1 for lustre-MDT0001, updating log [14056.889375] Lustre: 344122:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14057.270250] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14062.581606] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14063.064103] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [14064.671626] LustreError: 355006:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14064.681048] LustreError: 355006:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14064.703459] LustreError: 344126:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14064.712885] LustreError: 344126:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14064.831322] LustreError: 355012:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [14064.840183] LustreError: 355012:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [14064.841948] LustreError: 355012:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [14064.846189] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.141@tcp [14064.846189] [14369.765947] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14369.778222] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14369.816335] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14369.819250] Lustre: Skipped 3 previous similar messages [14375.571081] Lustre: server umount lustre-MDT0001 complete [14375.613423] LustreError: 345100:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [14375.634430] LustreError: 345100:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [14388.765563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14388.986687] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14388.991066] Lustre: Found index 1 for lustre-MDT0001, updating log [14389.022562] Lustre: 344122:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14389.372318] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14393.802967] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [14394.872394] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14397.948110] LustreError: 362796:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14397.960432] LustreError: 362796:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14397.975797] LustreError: 344126:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14397.981660] LustreError: 344126:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14398.027984] LustreError: 362799:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14398.054143] LustreError: 362799:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -114 [14398.062046] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14398.062046] [14716.897977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14716.906780] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14716.924870] Lustre: Skipped 1 previous similar message [14716.944448] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14716.955919] Lustre: Skipped 3 previous similar messages [14723.192738] Lustre: server umount lustre-MDT0000 complete [14727.654152] LustreError: 344135:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14727.676124] LustreError: 344135:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [14728.773043] LustreError: 362398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779141041 with bad export cookie 6382329909321488276 [14728.779843] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14728.784210] LustreError: 362398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14732.771099] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14732.778572] Lustre: Skipped 2 previous similar messages [14735.324835] Lustre: server umount lustre-MDT0001 complete [14740.407190] Lustre: server umount lustre-OST0000 complete [14751.711275] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [14762.659415] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [14804.396838] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 17:51:56 (1779141116) [14813.171345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14813.537530] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14813.608705] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14813.784524] Lustre: lustre-MDT0000: new disk, initializing [14813.937834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14813.974572] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14819.815975] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [14832.605824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14832.760592] Lustre: 372857:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14832.870241] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14832.880922] Lustre: Skipped 1 previous similar message [14832.973421] Lustre: lustre-MDT0001: new disk, initializing [14833.078310] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14833.135466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14833.149339] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14838.740909] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [14848.057358] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14853.454557] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14861.711572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14862.112056] Lustre: lustre-OST0000: new disk, initializing [14862.121142] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14862.143777] Lustre: 374598:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14862.311985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14869.135412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14869.157338] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14869.364525] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14870.568390] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [14881.913534] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14894.570736] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14894.577804] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14894.586938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14894.598913] Lustre: Skipped 8 previous similar messages [14897.620881] Lustre: server umount lustre-MDT0000 complete [14900.200434] LustreError: 372868:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14900.214414] LustreError: 372868:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [14902.300088] LustreError: 373644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1779141215 with bad export cookie 6382329909321490229 [14902.302670] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14902.313537] LustreError: 373644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14902.886376] Lustre: server umount lustre-MDT0001 complete [14917.921358] Lustre: server umount lustre-OST0000 complete [14950.359117] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [14961.736367] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [14996.609278] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14716 sec ========= 17:55:08 (1779141308) [14998.905160] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 17:55:10 (1779141310) === [15003.357811] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 17:55:14 (1779141314) === [15051.337537] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [15055.080764] Key type lgssc unregistered [15055.436458] LNet: 380817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15055.451743] LNetError: 380817:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [15055.471136] LNet: Removed LNI 192.168.201.141@tcp [15056.588922] Key type .llcrypt unregistered [15056.591815] Key type ._llcrypt unregistered