[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 569943320 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003231] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005010] kvm-guest: setup PV IPIs [ 0.008705] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011124] LSM: Security Framework initializing [ 0.012040] Yama: becoming mindful. [ 0.013029] SELinux: Initializing. [ 0.014050] *** VALIDATE selinux *** [ 0.022213] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026569] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027138] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028092] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029104] *** VALIDATE tmpfs *** [ 0.031104] *** VALIDATE proc *** [ 0.032219] *** VALIDATE cgroup *** [ 0.033006] *** VALIDATE cgroup2 *** [ 0.034245] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036012] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038027] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.042136] debug: unmapping init [mem 0xffffffffb0c59000-0xffffffffb0c60fff] [ 0.044981] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045638] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046019] ... version: 2 [ 0.047010] ... bit width: 48 [ 0.048009] ... generic registers: 4 [ 0.049009] ... value mask: 0000ffffffffffff [ 0.050010] ... max period: 00007fffffffffff [ 0.051009] ... fixed-purpose events: 3 [ 0.052011] ... event mask: 000000070000000f [ 0.053261] rcu: Hierarchical SRCU implementation. [ 0.055441] smp: Bringing up secondary CPUs ... [ 0.056517] x86: Booting SMP configuration: [ 0.057018] .... node #0, CPUs: #1 #2 #3 [ 0.063059] smp: Brought up 1 node, 4 CPUs [ 0.065016] smpboot: Max logical packages: 1 [ 0.066012] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.104022] node 0 deferred pages initialised in 36ms [ 0.109173] devtmpfs: initialized [ 0.110208] x86/mm: Memory block size: 128MB [ 0.112974] gcov: version magic: 0x41383552 [ 0.114326] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.115078] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.116222] pinctrl core: initialized pinctrl subsystem [ 0.117192] [ 0.117697] ************************************************************* [ 0.118016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.119012] ** ** [ 0.120013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.121012] ** ** [ 0.122008] ** This means that this kernel is built to expose internal ** [ 0.123012] ** IOMMU data structures, which may compromise security on ** [ 0.124010] ** your system. ** [ 0.125013] ** ** [ 0.126010] ** If you see this message and you are not debugging the ** [ 0.127011] ** kernel, report this immediately to your vendor! ** [ 0.128011] ** ** [ 0.129011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.130018] ************************************************************* [ 0.131797] NET: Registered protocol family 16 [ 0.132423] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.133057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.134062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.135429] cpuidle: using governor menu [ 0.137785] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.141335] PCI: Using configuration type 1 for base access [ 0.144122] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.156040] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.157013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.158313] cryptd: max_cpu_qlen set to 1000 [ 0.159437] ACPI: Added _OSI(Module Device) [ 0.160012] ACPI: Added _OSI(Processor Device) [ 0.161012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.162012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.166276] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.170221] ACPI: Interpreter enabled [ 0.171054] ACPI: PM: (supports S0 S3 S4 S5) [ 0.172011] ACPI: Using IOAPIC for interrupt routing [ 0.173130] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.174378] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.185934] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.186031] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.187016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.188071] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.190504] acpiphp: Slot [2] registered [ 0.191129] acpiphp: Slot [5] registered [ 0.192105] acpiphp: Slot [6] registered [ 0.193102] acpiphp: Slot [7] registered [ 0.194102] acpiphp: Slot [8] registered [ 0.195088] acpiphp: Slot [9] registered [ 0.196136] acpiphp: Slot [10] registered [ 0.197119] acpiphp: Slot [3] registered [ 0.198121] acpiphp: Slot [4] registered [ 0.199079] acpiphp: Slot [11] registered [ 0.200135] acpiphp: Slot [12] registered [ 0.201087] acpiphp: Slot [13] registered [ 0.202104] acpiphp: Slot [14] registered [ 0.203080] acpiphp: Slot [15] registered [ 0.204098] acpiphp: Slot [16] registered [ 0.205155] acpiphp: Slot [17] registered [ 0.206077] acpiphp: Slot [18] registered [ 0.207116] acpiphp: Slot [19] registered [ 0.208070] acpiphp: Slot [20] registered [ 0.209075] acpiphp: Slot [21] registered [ 0.210093] acpiphp: Slot [22] registered [ 0.211071] acpiphp: Slot [23] registered [ 0.212073] acpiphp: Slot [24] registered [ 0.213070] acpiphp: Slot [25] registered [ 0.214098] acpiphp: Slot [26] registered [ 0.215090] acpiphp: Slot [27] registered [ 0.216134] acpiphp: Slot [28] registered [ 0.218095] acpiphp: Slot [29] registered [ 0.221109] acpiphp: Slot [30] registered [ 0.222095] acpiphp: Slot [31] registered [ 0.224093] PCI host bridge to bus 0000:00 [ 0.226017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.230020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.233021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.237089] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.240020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.244021] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.247176] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.250496] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.254488] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.265829] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.271058] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.274019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.276015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.280019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.282576] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.284904] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.288044] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.290432] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.294014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.306948] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.312014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.319958] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.330000] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.340069] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.372000] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.381417] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.397013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.413012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.439019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.450000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.464018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.472013] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.496015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.505155] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.514015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.524021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.539022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.547000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.555013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.561000] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.580016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.590583] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.597016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.603018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.618028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.628641] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.632345] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.636423] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.640268] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.642204] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.647137] iommu: Default domain type: Passthrough [ 0.649486] SCSI subsystem initialized [ 0.651137] ACPI: bus type USB registered [ 0.653112] usbcore: registered new interface driver usbfs [ 0.655079] usbcore: registered new interface driver hub [ 0.658081] usbcore: registered new device driver usb [ 0.660167] pps_core: LinuxPPS API ver. 1 registered [ 0.662011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.666096] PTP clock support registered [ 0.669046] EDAC MC: Ver: 3.0.0 [ 0.671123] PCI: Using ACPI for IRQ routing [ 0.672626] NetLabel: Initializing [ 0.674016] NetLabel: domain hash size = 128 [ 0.676011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.678174] NetLabel: unlabeled traffic allowed by default [ 0.681150] vgaarb: loaded [ 0.683296] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.685012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.689013] clocksource: Switched to clocksource kvm-clock [ 0.815037] VFS: Disk quotas dquot_6.6.0 [ 0.816240] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.818308] *** VALIDATE ramfs *** [ 0.819278] *** VALIDATE hugetlbfs *** [ 0.820628] pnp: PnP ACPI init [ 0.822686] pnp: PnP ACPI: found 6 devices [ 0.839298] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.841689] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.843557] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.845390] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.847138] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.849080] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.851166] NET: Registered protocol family 2 [ 0.853468] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.857633] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.860542] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.864843] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.867646] TCP: Hash tables configured (established 65536 bind 65536) [ 0.870640] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.873171] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.875334] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.877590] NET: Registered protocol family 1 [ 0.879476] RPC: Registered named UNIX socket transport module. [ 0.881169] RPC: Registered udp transport module. [ 0.882456] RPC: Registered tcp transport module. [ 0.883724] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.885613] NET: Registered protocol family 44 [ 0.886914] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.888644] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.890300] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.892038] PCI: CLS 0 bytes, default 64 [ 0.893361] Unpacking initramfs... [ 2.292570] debug: unmapping init [mem 0xffff9e247cc54000-0xffff9e247ffbffff] [ 2.296319] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.298123] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.300330] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.829838] Initialise system trusted keyrings [ 2.831209] Key type blacklist registered [ 2.834367] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.842805] zbud: loaded [ 2.845531] *** VALIDATE nfs *** [ 2.846488] *** VALIDATE nfs4 *** [ 2.847966] pstore: using deflate compression [ 2.851028] Platform Keyring initialized [ 2.965988] NET: Registered protocol family 38 [ 2.968335] Key type asymmetric registered [ 2.970142] Asymmetric key parser 'x509' registered [ 2.972622] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.976732] io scheduler mq-deadline registered [ 2.978603] io scheduler kyber registered [ 2.982254] io scheduler bfq registered [ 2.984646] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.987155] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.989464] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.991822] ACPI: Power Button [PWRF] [ 2.996714] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.004485] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.032349] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.038851] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.050949] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.080749] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.120424] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.125290] Non-volatile memory driver v1.3 [ 3.127281] Linux agpgart interface v0.103 [ 3.158598] virtio_blk virtio1: [vda] 145880 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.162984] vda: detected capacity change from 0 to 74690560 [ 3.179308] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.183673] vdb: detected capacity change from 0 to 1073741824 [ 3.197788] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.201477] vdc: detected capacity change from 0 to 2621440000 [ 3.213844] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.217166] vdd: detected capacity change from 0 to 2621440000 [ 3.235803] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.239982] vde: detected capacity change from 0 to 4294967296 [ 3.255364] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.259656] vdf: detected capacity change from 0 to 4294967296 [ 3.273316] libphy: Fixed MDIO Bus: probed [ 3.280071] usbcore: registered new interface driver usbserial_generic [ 3.282523] usbserial: USB Serial support registered for generic [ 3.284949] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.289896] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.291740] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.294299] mousedev: PS/2 mouse device common for all mice [ 3.297804] rtc_cmos 00:05: RTC can wake from S4 [ 3.301138] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.308380] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.309558] rtc_cmos 00:05: registered as rtc0 [ 3.314739] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.315489] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.318441] intel_pstate: CPU model not supported [ 3.326312] hid: raw HID events driver (C) Jiri Kosina [ 3.329161] usbcore: registered new interface driver usbhid [ 3.331568] usbhid: USB HID core driver [ 3.333635] drop_monitor: Initializing network drop monitor service [ 3.336544] Initializing XFRM netlink socket [ 3.338593] NET: Registered protocol family 10 [ 3.341792] Segment Routing with IPv6 [ 3.343303] NET: Registered protocol family 17 [ 3.345371] mpls_gso: MPLS GSO support [ 3.350464] RAS: Correctable Errors collector initialized. [ 3.352675] AVX version of gcm_enc/dec engaged. [ 3.354421] AES CTR mode by8 optimization enabled [ 3.440585] sched_clock: Marking stable (3440554382, 0)->(4494967776, -1054413394) [ 3.445495] registered taskstats version 1 [ 3.448270] Loading compiled-in X.509 certificates [ 3.451053] zswap: loaded using pool lzo/zbud [ 3.474777] Key type big_key registered [ 3.489132] Key type encrypted registered [ 3.490751] ima: No TPM chip found, activating TPM-bypass! [ 3.492811] ima: Allocated hash algorithm: sha1 [ 3.494691] ima: No architecture policies found [ 3.496750] evm: Initialising EVM extended attributes: [ 3.498709] evm: security.selinux [ 3.499981] evm: security.ima [ 3.501096] evm: security.capability [ 3.502402] evm: HMAC attrs: 0x1 [ 3.504759] rtc_cmos 00:05: setting system clock to 2026-07-29 07:26:55 UTC (1785310015) [ 3.511772] debug: unmapping init [mem 0xffffffffb1c03000-0xffffffffb1dfffff] [ 3.514855] debug: unmapping init [mem 0xffffffffb0982000-0xffffffffb0c58fff] [ 3.523497] Write protecting the kernel read-only data: 28672k [ 3.527244] debug: unmapping init [mem 0xffffffffaf003000-0xffffffffaf1fffff] [ 3.529949] debug: unmapping init [mem 0xffffffffaf914000-0xffffffffaf9fffff] [ 3.565585] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.574220] systemd[1]: Detected virtualization kvm. [ 3.576223] systemd[1]: Detected architecture x86-64. [ 3.578079] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.605852] systemd[1]: No hostname configured. [ 3.607581] systemd[1]: Set hostname to . [ 3.612088] random: systemd: uninitialized urandom read (16 bytes read) [ 3.614451] systemd[1]: Initializing machine ID from random generator. [ 3.665411] random: ln: uninitialized urandom read (6 bytes read) [ 3.775916] random: systemd: uninitialized urandom read (16 bytes read) [ 3.778747] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.783767] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.789442] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Journal Service. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.565143] device-mapper: uevent: version 1.0.3 [ 4.566898] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK [ 5.127036] random: fast init done ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.335806] virtio_net virtio0 ens2: renamed from eth0 [ 5.379752] scsi host0: ata_piix [ 5.410500] scsi host1: ata_piix [ 5.411906] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.414329] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.191359] dracut-initqueue[585]: RTNETLINK answers: File exists [ 10.006643] random: crng init done [ 10.007925] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.517522] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.689988] printk: systemd: 20 output lines suppressed due to ratelimiting [ 11.992167] SELinux: Disabled at runtime. [ 12.058175] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.066153] systemd[1]: Detected virtualization kvm. [ 12.067755] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.607965] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.611573] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.617429] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.622282] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.628257] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.639872] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.646586] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. Starting udev Coldplug all Devices... [ OK ] Listening on Process Core Dump Socket. Mounting Huge Pages File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Reached target RPC Port Mapper. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ 12.899893] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 13.185589] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.514494] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.531607] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.634594] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.649389] EDAC sbridge: Ver: 1.1.2 [ 15.427555] Key type dns_resolver registered [ 15.742420] NFS: Registering the id_resolver key type [ 15.745086] Key type id_resolver registered [ 15.747189] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg431-server login: [ 47.717751] libcfs: loading out-of-tree module taints kernel. [ 47.827651] Key type ._llcrypt registered [ 47.831470] Key type .llcrypt registered [ 48.026583] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 79.952653] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 82.599246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 82.633215] alg: No test for adler32 (adler32-zlib) [ 84.337511] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 86.226546] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 88.159319] Key type lgssc registered [ 91.007872] Lustre: Echo OBD driver; http://www.lustre.org/ [ 118.603856] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 172.280688] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 192.971620] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 193.039443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 194.589579] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 194.698777] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 194.975321] Lustre: lustre-MDT0000: new disk, initializing [ 195.119285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.146048] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 201.581802] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.581046] hrtimer: interrupt took 4028122 ns [ 218.834726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 218.948891] Lustre: 6518:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 218.984777] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 218.989615] Lustre: Skipped 1 previous similar message [ 219.077740] Lustre: lustre-MDT0001: new disk, initializing [ 219.201245] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 219.240856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 219.255324] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 226.297846] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.147969] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 246.657277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 247.205391] Lustre: lustre-OST0000: new disk, initializing [ 247.214784] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 247.238277] Lustre: 8455:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 247.361415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 255.581403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 255.605067] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 255.676107] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 256.596673] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 276.847108] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 277.164308] Lustre: lustre-OST0001: new disk, initializing [ 277.189792] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 277.206924] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 277.403718] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 286.779483] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 286.794774] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 286.957364] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 288.847652] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 304.982350] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 315.853501] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 327.552509] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing check_logdir /tmp/testlogs/ [ 337.556579] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing yml_node [ 347.470837] Lustre: DEBUG MARKER: Client: 2.17.55.22 [ 352.042817] Lustre: DEBUG MARKER: MDS: 2.17.55.22 [ 356.875249] Lustre: DEBUG MARKER: OSS: 2.17.55.22 [ 359.924530] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jul 29 03:32:48 EDT 2026 [ 393.076146] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 395.910917] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 415.200276] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 415.208934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 415.230074] Lustre: Skipped 2 previous similar messages [ 415.254853] Lustre: Skipped 3 previous similar messages [ 420.362333] Lustre: server umount lustre-MDT0000 complete [ 425.279955] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310437 with bad export cookie 6679877265816985899 [ 425.288272] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 425.303027] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 425.440276] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 425.454788] LustreError: 8755:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 425.466610] Lustre: Skipped 1 previous similar message [ 425.488607] LustreError: 8755:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 425.510025] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 430.560969] LustreError: 6525:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 430.561391] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 430.591694] Lustre: Skipped 2 previous similar messages [ 432.222612] Lustre: server umount lustre-MDT0001 complete [ 438.320965] Lustre: server umount lustre-OST0000 complete [ 444.025276] Lustre: server umount lustre-OST0001 complete [ 453.720674] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 470.074140] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 522.826593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 523.073960] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 523.096961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 523.173311] Lustre: lustre-MDT0000: new disk, initializing [ 523.300975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 523.334980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 531.027646] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 544.111956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.373223] Lustre: 15113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 544.465686] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 544.472005] Lustre: Skipped 1 previous similar message [ 544.680557] Lustre: lustre-MDT0001: new disk, initializing [ 544.835328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 544.911383] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 544.922937] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 551.408615] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 563.304793] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 570.390640] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 579.222974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 579.703554] Lustre: lustre-OST0000: new disk, initializing [ 579.709957] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 579.716645] Lustre: 17084:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 579.840425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 581.586726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 581.600575] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 581.767424] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 588.068868] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 599.949844] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 607.197882] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 607.462770] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 613.500480] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 613.760024] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 617.446221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 617.447505] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 617.465898] Lustre: Skipped 1 previous similar message [ 617.482699] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 622.565418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 622.585501] Lustre: Skipped 1 previous similar message [ 630.239186] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 630.966323] Lustre: server umount lustre-OST0000 complete [ 637.815379] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 637.839657] Lustre: Skipped 1 previous similar message [ 637.853854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 637.866571] Lustre: Skipped 4 previous similar messages [ 652.255198] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 652.989239] Lustre: server umount lustre-MDT0000 complete [ 653.290509] LustreError: 15126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 653.316907] LustreError: 15126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 657.376797] LustreError: 15127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 657.389305] LustreError: 15127:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 657.999535] LustreError: 15107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310669 with bad export cookie 6679877265816987432 [ 658.005306] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 658.013096] LustreError: 15107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 658.794253] Lustre: server umount lustre-MDT0001 complete [ 665.711766] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:37:54 (1785310674) [ 676.253364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 677.413819] LustreError: 19353:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 677.646712] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 683.099585] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 687.076192] LustreError: 19354:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 687.098773] LustreError: 19354:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 694.993235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 703.878827] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 714.666930] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 722.993292] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 730.865772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 739.959759] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 751.127872] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 756.371153] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 767.465110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 767.474668] LustreError: Skipped 1 previous similar message [ 767.480468] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 767.497508] Lustre: Skipped 1 previous similar message [ 767.503989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 767.513042] Lustre: Skipped 4 previous similar messages [ 771.401924] Lustre: server umount lustre-MDT0000 complete [ 773.614074] LustreError: 19350:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 773.630542] LustreError: 19350:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 776.633997] LustreError: 19334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310788 with bad export cookie 6679877265816988307 [ 776.651939] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 776.653961] LustreError: 19334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 777.238719] Lustre: server umount lustre-MDT0001 complete [ 792.829494] Lustre: server umount lustre-OST0000 complete [ 820.158409] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 832.930927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.454607] LustreError: 23741:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 833.544178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 833.556090] Lustre: Skipped 2 previous similar messages [ 838.325440] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 849.647435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.641124] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 860.477768] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 869.681805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 879.810876] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 891.456032] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 891.852508] Lustre: lustre-OST0001: new disk, initializing [ 891.860235] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 891.875900] Lustre: 26057:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 893.071806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 893.086776] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 893.270404] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 901.304485] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 915.406407] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 921.397644] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 939.493434] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 939.496179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 939.526842] Lustre: Skipped 4 previous similar messages [ 939.548946] Lustre: Skipped 6 previous similar messages [ 944.582751] Lustre: server umount lustre-MDT0000 complete [ 944.612411] LustreError: 23741:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 944.629484] LustreError: 23741:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 948.675360] LustreError: 23722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785310960 with bad export cookie 6679877265816989161 [ 948.683714] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 948.715181] LustreError: 23722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 949.529758] Lustre: server umount lustre-MDT0001 complete [ 965.261509] Lustre: server umount lustre-OST0000 complete [ 965.922048] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785310961/real 1785310961] req@ffff9e24fbb9fb80 x1872033321608576/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785310977 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 965.952378] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 965.968833] Lustre: Skipped 1 previous similar message [ 969.184160] Lustre: 3650:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785310965/real 1785310965] req@ffff9e24fbb9e680 x1872033321608832/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785310981 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 970.147585] Lustre: server umount lustre-OST0001 complete [ 995.594727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1006.635653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1007.101942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1007.116852] Lustre: Skipped 3 previous similar messages [ 1012.445413] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1023.595703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1031.253197] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1034.876165] Lustre: 30231:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1044.074518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1054.443688] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1066.533533] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1074.988363] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1083.950846] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1087.854873] Lustre: 32111:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1096.166766] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1096.173584] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1096.188834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1100.853836] Lustre: server umount lustre-MDT0000 complete [ 1102.837474] LustreError: 29082:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1102.851611] LustreError: 29082:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 1105.645303] LustreError: 29063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311117 with bad export cookie 6679877265816990456 [ 1105.662144] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1105.680983] LustreError: 29063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1106.310373] Lustre: server umount lustre-MDT0001 complete [ 1121.204541] Lustre: server umount lustre-OST0000 complete [ 1123.301404] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311119/real 1785311119] req@ffff9e24fbb9d880 x1872033321665664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311135 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1126.437296] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311122/real 1785311122] req@ffff9e24f6087800 x1872033321665920/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311138 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1127.290413] Lustre: server umount lustre-OST0001 complete [ 1139.855353] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:45:48 (1785311148) [ 1174.965501] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1186.578965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1192.995444] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1202.573044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1207.671860] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1211.557179] Lustre: 36061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1219.586496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1228.261490] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1240.501598] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1249.846206] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1258.588770] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1262.736704] Lustre: 37939:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1271.781524] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1271.785372] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1271.808962] Lustre: Skipped 6 previous similar messages [ 1271.827768] Lustre: Skipped 7 previous similar messages [ 1276.632101] Lustre: server umount lustre-MDT0000 complete [ 1280.900438] LustreError: 36064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311292 with bad export cookie 6679877265816991562 [ 1280.907943] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1280.918442] LustreError: 36064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1281.805421] Lustre: server umount lustre-MDT0001 complete [ 1297.248295] Lustre: server umount lustre-OST0000 complete [ 1298.209572] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311294/real 1785311294] req@ffff9e24fc3fe300 x1872033321720576/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311310 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1302.337573] Lustre: server umount lustre-OST0001 complete [ 1313.019664] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:48:42 (1785311322) [ 1334.646406] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1348.410480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1349.102343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1349.113866] Lustre: Skipped 7 previous similar messages [ 1354.613090] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1359.332121] LustreError: 40081:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1359.350085] LustreError: 40081:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 1365.534221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1373.265818] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1376.999086] Lustre: 41234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1385.997067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1395.716133] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1407.955839] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1416.543162] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1426.312645] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1430.926720] Lustre: 43112:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1432.214616] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1432.223923] Lustre: Skipped 1 previous similar message [ 1437.664352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1437.692857] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1437.719342] Lustre: Skipped 2 previous similar messages [ 1442.703695] Lustre: server umount lustre-MDT0000 complete [ 1447.393677] LustreError: 40062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311459 with bad export cookie 6679877265816992668 [ 1447.417311] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1453.788144] Lustre: server umount lustre-MDT0001 complete [ 1463.429157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1478.964530] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1482.039310] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1491.973420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1498.113483] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1498.125187] Lustre: Skipped 1 previous similar message [ 1499.370215] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1507.637439] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1509.973348] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1518.130759] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1520.630370] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1533.940138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1533.948977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1533.952981] Lustre: Skipped 16 previous similar messages [ 1536.793121] Lustre: server umount lustre-MDT0000 complete [ 1541.458813] LustreError: 40061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311553 with bad export cookie 6679877265816993767 [ 1541.460065] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1541.471899] LustreError: 40061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1542.293134] Lustre: server umount lustre-MDT0001 complete [ 1557.678621] Lustre: server umount lustre-OST0000 complete [ 1560.351127] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311556/real 1785311556] req@ffff9e24fc9aa680 x1872033321824128/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311572 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1562.448359] Lustre: server umount lustre-OST0001 complete [ 1572.303653] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:53:01 (1785311581) [ 1590.027880] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1601.784724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1608.000100] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1618.831069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1625.188905] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1628.267322] Lustre: 48871:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1636.000941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1644.914987] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1654.776395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1663.399526] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1672.906392] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1678.047374] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1701.364058] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1701.383765] Lustre: Skipped 11 previous similar messages [ 1701.394524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1704.130360] Lustre: server umount lustre-MDT0000 complete [ 1708.634212] LustreError: 47702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785311720 with bad export cookie 6679877265816994586 [ 1708.635348] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1708.647282] LustreError: 47702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1709.057314] Lustre: server umount lustre-MDT0001 complete [ 1723.839881] Lustre: server umount lustre-OST0000 complete [ 1727.521784] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785311723/real 1785311723] req@ffff9e23c3035180 x1872033321888640/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785311739 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1727.968409] Lustre: server umount lustre-OST0001 complete [ 1738.378058] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:55:47 (1785311747) [ 1740.948718] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1742.692706] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:55:52 (1785311752) [ 1756.432867] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 1767.851487] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1794.427517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1794.875754] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1794.979453] Lustre: lustre-MDT0000: new disk, initializing [ 1795.161925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1801.296727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1811.552213] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1819.405933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1819.793381] Lustre: lustre-OST0000: new disk, initializing [ 1819.801838] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1819.806170] Lustre: Skipped 1 previous similar message [ 1819.819926] Lustre: 54972:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1821.108741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1821.122453] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1821.205498] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1826.684145] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1835.939973] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1838.620467] Lustre: DEBUG MARKER: create test files [ 1877.477790] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1880.591658] Lustre: server umount lustre-OST0000 complete [ 1884.919789] Lustre: server umount lustre-MDT0000 complete [ 1897.594332] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 1901.475739] Key type lgssc unregistered [ 1901.899527] LNet: 56600:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1901.909760] LNetError: 56600:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1901.930294] LNet: Removed LNI 192.168.204.131@tcp [ 1903.129160] Key type .llcrypt unregistered [ 1903.131907] Key type ._llcrypt unregistered [ 1933.057951] Key type ._llcrypt registered [ 1933.059939] Key type .llcrypt registered [ 1933.130609] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 1934.258557] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1934.316662] alg: No test for adler32 (adler32-zlib) [ 1935.328284] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 1935.518422] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 1937.183193] Key type lgssc registered [ 1938.529614] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1946.964199] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1946.992282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1948.578365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1953.880770] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1961.848756] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1969.172292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1969.503252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1973.625018] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1973.734317] Lustre: 59334:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1973.747830] Lustre: 59334:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1973.759952] Lustre: 59334:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1973.773233] Lustre: 59334:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1973.787100] Lustre: 59334:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1973.798328] Lustre: 59334:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1976.381415] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 1983.733048] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1989.149385] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1994.393629] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2004.448239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2004.475609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2009.568442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2014.691373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2016.223147] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2016.476510] Lustre: server umount lustre-OST0000 complete [ 2020.660329] Lustre: server umount lustre-MDT0000 complete [ 2031.123538] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 2034.900398] Key type lgssc unregistered [ 2035.269966] LNet: 60944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2035.275749] LNetError: 60944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2035.297253] LNet: Removed LNI 192.168.204.131@tcp [ 2036.195250] Key type .llcrypt unregistered [ 2036.196993] Key type ._llcrypt unregistered [ 2067.740782] Key type ._llcrypt registered [ 2067.744709] Key type .llcrypt registered [ 2067.894972] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2069.100978] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2069.208015] alg: No test for adler32 (adler32-zlib) [ 2070.441770] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 2070.740842] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 2072.399255] Key type lgssc registered [ 2073.500907] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2080.737850] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2080.750995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2082.209898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2086.443763] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2094.087535] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2101.589223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2101.987394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2107.131293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2107.250228] Lustre: 63793:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2107.265201] Lustre: 63793:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2107.273926] Lustre: 63793:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2107.297995] Lustre: 63793:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2107.303601] Lustre: 63793:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2107.314543] Lustre: 63793:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2109.064442] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2117.209468] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2123.636166] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2132.961042] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2132.968122] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2132.984039] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2138.080853] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2139.081365] Lustre: server umount lustre-OST0000 complete [ 2143.282413] Lustre: server umount lustre-MDT0000 complete [ 2156.488738] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 2159.440516] Key type lgssc unregistered [ 2159.695733] LNet: 65350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2159.703168] LNetError: 65350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2159.724607] LNet: Removed LNI 192.168.204.131@tcp [ 2160.858885] Key type .llcrypt unregistered [ 2160.863863] Key type ._llcrypt unregistered [ 2179.704939] Key type ._llcrypt registered [ 2179.712961] Key type .llcrypt registered [ 2179.824538] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 2196.146599] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2197.382395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2197.431071] alg: No test for adler32 (adler32-zlib) [ 2198.522948] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 2198.694261] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 2200.392091] Key type lgssc registered [ 2202.176248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2239.478262] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2239.534328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2240.825922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2240.847285] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2240.923609] Lustre: lustre-MDT0000: new disk, initializing [ 2241.012109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2241.033411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2245.730527] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2256.673847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2256.821227] Lustre: 68506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2256.953963] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2256.961150] Lustre: Skipped 1 previous similar message [ 2257.090677] Lustre: lustre-MDT0001: new disk, initializing [ 2257.240113] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2257.299863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2257.318563] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2262.152897] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2274.156880] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2281.783597] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2290.648420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2291.043301] Lustre: lustre-OST0000: new disk, initializing [ 2291.048640] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2291.055302] Lustre: 70473:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2291.158184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2296.888192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2296.900953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2296.991605] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2297.300311] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2307.298773] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2313.561729] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2313.737856] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2319.567216] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2319.813352] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2322.912753] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2322.917081] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2322.930329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2322.942255] Lustre: Skipped 1 previous similar message [ 2322.957043] Lustre: Skipped 1 previous similar message [ 2328.033179] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2328.046502] Lustre: Skipped 1 previous similar message [ 2328.363409] Lustre: server umount lustre-OST0000 complete [ 2333.153703] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2333.157971] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2333.177689] Lustre: Skipped 1 previous similar message [ 2333.195409] Lustre: Skipped 1 previous similar message [ 2338.273069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2338.280566] Lustre: Skipped 1 previous similar message [ 2338.918352] Lustre: server umount lustre-MDT0000 complete [ 2342.630276] LustreError: 68497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312354 with bad export cookie 14422015291106414346 [ 2342.641859] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2343.096565] Lustre: server umount lustre-MDT0001 complete [ 2350.876679] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:06:00 (1785312360) [ 2361.584466] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2399.908404] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 2407.470388] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2439.448259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2439.626687] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2439.650117] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2439.710869] Lustre: lustre-MDT0000: new disk, initializing [ 2439.761668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2439.777420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2444.541692] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2455.579906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2455.741397] Lustre: 76493:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2455.810918] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2455.816426] Lustre: Skipped 1 previous similar message [ 2455.919615] Lustre: lustre-MDT0001: new disk, initializing [ 2456.054495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2456.112929] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2456.132795] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2460.593565] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2470.351664] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2475.974731] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2483.143978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2483.576161] Lustre: lustre-OST0000: new disk, initializing [ 2483.580467] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2483.585964] Lustre: 78461:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2483.726200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2490.076641] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2494.051648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2494.069507] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2494.203335] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2501.737614] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2506.352367] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2506.650622] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2512.049458] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2512.269239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2514.924378] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2514.929480] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2514.938531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2514.945745] LustreError: Skipped 2 previous similar messages [ 2514.960424] Lustre: Skipped 1 previous similar message [ 2521.360620] Lustre: server umount lustre-OST0000 complete [ 2527.712912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2527.722083] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2527.731926] Lustre: Skipped 1 previous similar message [ 2527.741387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2527.743283] Lustre: Skipped 2 previous similar messages [ 2531.898897] Lustre: server umount lustre-MDT0000 complete [ 2535.496742] LustreError: 76485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312547 with bad export cookie 14422015291106415228 [ 2535.496970] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2535.507629] LustreError: 76485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2536.074276] Lustre: server umount lustre-MDT0001 complete [ 2544.508339] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:09:13 (1785312553) [ 2551.715344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2552.202095] LustreError: 80876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2552.217502] LustreError: 80876:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2552.300854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2556.757321] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2557.423449] LustreError: 80877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2562.537865] LustreError: 80876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2564.834986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2569.764769] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2578.065874] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2585.165873] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2593.039643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2593.563934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2593.571419] Lustre: Skipped 1 previous similar message [ 2600.736471] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2608.052446] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2611.362713] Lustre: 83548:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 22267b78-a758-488c-ad2a-ad0466766453 at adminstrative request [ 2616.296713] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2616.311312] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2626.550494] Lustre: MGS: Client 22267b78-a758-488c-ad2a-ad0466766453 (at 0@lo) reconnecting [ 2626.562642] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xc8254a7f03a655ee to 0xc8254a7f03a6588e [ 2626.588783] Lustre: MGC192.168.204.131@tcp: Connection restored to 0@lo (at 0@lo) [ 2648.018210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2648.223692] Lustre: lustre-OST0001: new disk, initializing [ 2648.228873] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2648.236405] Lustre: 83949:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2654.309075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2654.311744] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2654.324533] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2654.409207] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2663.782927] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2670.055724] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2670.056578] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2670.063552] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2670.063559] Lustre: Skipped 2 previous similar messages [ 2670.068030] Lustre: Skipped 2 previous similar messages [ 2670.092335] LustreError: Skipped 1 previous similar message [ 2672.955618] Lustre: server umount lustre-OST0001 complete [ 2675.190276] LustreError: 82699:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2675.214291] LustreError: 82699:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2677.728931] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2680.289559] LustreError: 82700:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2680.316905] LustreError: 82700:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2680.333376] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2680.353385] Lustre: Skipped 1 previous similar message [ 2690.550960] LustreError: 85005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2690.567135] LustreError: 85005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2691.039395] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2691.422797] Lustre: server umount lustre-OST0000 complete [ 2698.227643] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2698.233237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2702.273501] Lustre: server umount lustre-MDT0000 complete [ 2706.340377] LustreError: 80857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312718 with bad export cookie 14422015291106416782 [ 2706.343327] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2706.357454] LustreError: 80857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2706.698726] Lustre: server umount lustre-MDT0001 complete [ 2713.981887] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:12:03 (1785312723) [ 2715.156283] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2716.884660] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:12:06 (1785312726) [ 2718.299377] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2720.081619] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:12:09 (1785312729) [ 2721.580994] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2723.267432] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:12:12 (1785312732) [ 2724.784278] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2726.297271] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:12:16 (1785312736) [ 2728.124842] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2733.402775] loop: module loaded [ 2733.423657] loop0: detected capacity change from 0 to 4294967296 [ 2733.478942] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2735.054956] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2738.767222] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2743.096381] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2768.257103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2768.302603] LustreError: 88072:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2768.310367] LustreError: 88072:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2768.316903] LustreError: 88072:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2768.327802] LustreError: 88072:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2788.150348] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 2797.681242] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 2831.710347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2831.917419] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2831.943670] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2832.023504] Lustre: lustre-MDT0000: new disk, initializing [ 2832.075161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2832.078180] Lustre: Skipped 1 previous similar message [ 2832.085640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2836.414200] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2846.981720] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2847.101142] Lustre: 90936:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2847.386674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2851.894873] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2860.900780] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2865.861627] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2872.176956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2872.610649] Lustre: 92903:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2878.744883] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 2881.099992] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2881.263818] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2890.230573] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2899.285677] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2899.989451] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2912.262793] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2912.886402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2917.345250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2917.350077] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2917.356685] LustreError: Skipped 1 previous similar message [ 2917.359294] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2917.375682] Lustre: Skipped 2 previous similar messages [ 2917.414692] Lustre: Skipped 12 previous similar messages [ 2923.053470] Lustre: server umount lustre-OST0000 complete [ 2935.813948] LustreError: 90948:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2935.844658] LustreError: 90948:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2936.720078] Lustre: server umount lustre-MDT0000 complete [ 2945.081190] LustreError: 90928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785312957 with bad export cookie 14422015291106417398 [ 2945.108112] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2946.074775] Lustre: server umount lustre-MDT0001 complete [ 2962.423257] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 04:16:10 (1785312970) [ 2974.272404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2974.970982] LustreError: 95320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2974.991301] LustreError: 95320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2975.045099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2975.051120] Lustre: Skipped 2 previous similar messages [ 2982.539695] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3005.378417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3013.068317] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3022.753791] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3030.417244] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3041.149362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3048.731955] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3058.494971] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3065.815349] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3066.170627] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3073.721901] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3074.091718] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3082.327893] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3082.577699] Lustre: lustre-OST0001: new disk, initializing [ 3082.582037] Lustre: Skipped 2 previous similar messages [ 3082.586728] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3082.591738] Lustre: Skipped 3 previous similar messages [ 3082.605262] Lustre: 98698:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3092.160466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3092.177519] Lustre: Skipped 2 previous similar messages [ 3092.189467] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3092.328908] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3092.726106] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3107.177783] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3114.022417] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3114.436737] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3121.544336] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3121.943134] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3146.329534] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3180.608043] Lustre: Failing over lustre-MDT0000 [ 3181.179476] Lustre: server umount lustre-MDT0000 complete [ 3181.790992] LustreError: 95315:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3181.818902] LustreError: 95315:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 3184.613879] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3184.645050] Lustre: Skipped 3 previous similar messages [ 3200.991922] Lustre: 66729:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785313196/real 1785313196] req@000000006660b636 x1872035536215296/t0(0) o400->MGC192.168.204.131@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785313212 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3201.030505] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3213.445351] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3213.448633] LDISKFS-fs (dm-0): recovery complete [ 3213.464501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3227.044597] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3227.844821] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3227.846290] Lustre: 98447:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3232.240283] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3232.252910] Lustre: 96238:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3232.276491] Lustre: 96238:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 3232.397057] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3232.721217] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3233.060312] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3233.079280] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3233.415088] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3233.422943] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3233.436991] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3233.458403] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3233.982177] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3234.129894] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3234.139589] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3234.487179] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3234.496623] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3235.489274] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3235.515032] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 3236.179271] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3236.189024] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3236.532391] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3236.548888] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3239.741047] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3239.757542] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 3240.385556] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3240.402314] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3240.727121] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3240.732614] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3248.001926] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3248.030114] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 3248.673742] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3248.695026] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 3249.024034] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3249.035954] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 3264.295960] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3264.323317] Lustre: 101557:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 3264.969645] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3264.979557] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3265.311607] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3265.326711] LustreError: 101557:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3287.064042] Lustre: 101557:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3287.122038] Lustre: 101557:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3287.168070] Lustre: 101557:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b809dce3-58bf-4ac9-a3fa-9a46eefdbb3f@192.168.204.31@tcp [ 3287.203422] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3287.212225] Lustre: 101557:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000b5883b72 x1872035518156416/t0(8589934767) o101->b809dce3-58bf-4ac9-a3fa-9a46eefdbb3f@192.168.204.31@tcp:294/0 lens 592/0 e 0 to 0 dl 1785313309 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3287.299082] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3287.307142] Lustre: lustre-MDT0000: Denying connection for new client b809dce3-58bf-4ac9-a3fa-9a46eefdbb3f (at 192.168.204.31@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3287.397022] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3287.432934] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3287.436589] Lustre: Skipped 2 previous similar messages [ 3287.442220] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3287.464896] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 3287.465217] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3312.611270] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3312.619514] LustreError: Skipped 1 previous similar message [ 3312.630647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3312.654700] Lustre: Skipped 2 previous similar messages [ 3312.658408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3312.669991] Lustre: Skipped 5 previous similar messages [ 3323.361823] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3323.909136] Lustre: server umount lustre-OST0000 complete [ 3324.398387] LustreError: 97147:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3324.416718] LustreError: 97147:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 58 previous similar messages [ 3339.123829] Lustre: server umount lustre-OST0001 complete [ 3351.810518] Lustre: server umount lustre-MDT0000 complete [ 3357.317336] LustreError: 95302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785313369 with bad export cookie 14422015291106468617 [ 3357.322992] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3357.333069] LustreError: 95302:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3358.039728] Lustre: server umount lustre-MDT0001 complete [ 3369.703461] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 04:22:59 (1785313379) [ 3379.666966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3379.691298] Lustre: *** cfs_fail_loc=197, val=0*** [ 3395.424288] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3400.544956] LustreError: 103556:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.131@tcp: failed processing log, type 4: rc = -110 [ 3405.789593] Lustre: *** cfs_fail_loc=197, val=0*** [ 3405.791440] Lustre: Skipped 65 previous similar messages [ 3426.401236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3426.408088] Lustre: Skipped 4 previous similar messages [ 3437.248581] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3449.724974] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3453.683990] Lustre: server umount lustre-OST0000 complete [ 3465.598639] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 04:24:34 (1785313474) [ 3493.179995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3499.979331] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3512.515567] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3517.632422] LustreError: 105210:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3517.662265] LustreError: 105210:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3517.680538] LustreError: 105210:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 23, retries 0, failed: rc = -5 [ 3518.663855] Lustre: server umount lustre-MDT0000 complete [ 3562.954834] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 3580.781369] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3632.047449] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3632.395421] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3632.451462] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3632.584287] Lustre: lustre-MDT0000: new disk, initializing [ 3632.748634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3641.014803] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3662.104758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3662.658299] Lustre: 109422:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3663.841955] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3672.031678] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3685.635704] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3693.280662] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3701.962799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3702.309260] Lustre: 111393:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3711.290987] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3711.878572] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3729.221298] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3736.991531] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3737.275224] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3745.916450] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3746.438197] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3752.423727] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3752.426182] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3752.434664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3752.434672] Lustre: Skipped 12 previous similar messages [ 3752.461599] Lustre: Skipped 6 previous similar messages [ 3752.518299] LustreError: Skipped 4 previous similar messages [ 3764.194533] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3764.705436] Lustre: server umount lustre-OST0000 complete [ 3778.530227] Lustre: server umount lustre-MDT0000 complete [ 3782.629380] LustreError: 109429:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3782.653986] LustreError: 109429:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 3784.136096] LustreError: 109416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785313796 with bad export cookie 14422015291106479166 [ 3784.158289] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3785.665356] Lustre: server umount lustre-MDT0001 complete [ 3798.931730] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 04:30:07 (1785313807) [ 3816.846263] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 3821.605971] Key type lgssc unregistered [ 3822.086568] LNet: 113907:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3822.103352] LNetError: 113907:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3823.152813] LNet: Removed LNI 192.168.204.131@tcp [ 3824.619568] Key type .llcrypt unregistered [ 3824.621661] Key type ._llcrypt unregistered [ 3851.569676] Key type ._llcrypt registered [ 3851.571338] Key type .llcrypt registered [ 3851.684302] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 3873.978301] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 3875.646812] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3875.854668] alg: No test for adler32 (adler32-zlib) [ 3877.175513] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 3877.803428] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 3879.649544] Key type lgssc registered [ 3881.531641] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3941.083793] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3941.109632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3942.441756] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3942.472158] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3942.582311] Lustre: lustre-MDT0000: new disk, initializing [ 3942.707991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3942.735391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3948.341107] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3961.643645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3962.136671] Lustre: lustre-OST0000: new disk, initializing [ 3962.143105] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3962.151498] Lustre: Skipped 1 previous similar message [ 3962.159244] Lustre: 118088:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3962.336225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3968.322669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3968.349178] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3968.462348] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3973.873263] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 3996.776282] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3997.167260] Lustre: lustre-OST0001: new disk, initializing [ 3997.188436] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3997.218153] Lustre: 118958:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3997.444250] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4004.136713] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4004.161466] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4004.232957] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4008.535093] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4024.801877] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4024.842894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4024.853576] Lustre: Skipped 1 previous similar message [ 4028.242447] Lustre: server umount lustre-MDT0000 complete [ 4038.837861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4044.813857] LustreError: 118956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314056 with bad export cookie 16293682502694284915 [ 4044.823564] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4044.827129] LustreError: 118956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4045.229688] Lustre: server umount lustre-OST0000 complete [ 4050.817122] Lustre: server umount lustre-OST0001 complete [ 4063.414206] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 04:34:32 (1785314072) [ 4096.905631] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 04:35:05 (1785314105) [ 4107.252577] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4138.556641] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 04:35:47 (1785314147) [ 4192.513782] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 4209.265410] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4261.363644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4261.926045] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4262.016982] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4262.262741] Lustre: lustre-MDT0000: new disk, initializing [ 4262.412332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4262.452594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4269.022363] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4282.317246] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4282.512432] Lustre: 126737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4282.608325] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4282.615218] Lustre: Skipped 1 previous similar message [ 4282.793831] Lustre: lustre-MDT0001: new disk, initializing [ 4282.976694] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4283.078480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4283.088859] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4289.532897] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4302.789773] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4312.997245] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4324.095116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4324.589268] Lustre: lustre-OST0000: new disk, initializing [ 4324.593659] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4324.618620] Lustre: 128709:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4324.794650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4330.113363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4330.140251] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4330.325053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4333.056338] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4345.563558] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4352.317189] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4352.594366] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4359.547597] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4359.866499] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4366.307919] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4366.309871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4366.346337] Lustre: Skipped 1 previous similar message [ 4366.364255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4366.374496] Lustre: Skipped 1 previous similar message [ 4369.191529] Lustre: server umount lustre-OST0000 complete [ 4375.010844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4375.020531] LustreError: Skipped 1 previous similar message [ 4375.026460] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4375.040094] Lustre: Skipped 1 previous similar message [ 4375.067289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4381.249764] Lustre: server umount lustre-MDT0000 complete [ 4386.805689] LustreError: 127779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4386.842283] LustreError: 127779:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4387.859899] LustreError: 127544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314399 with bad export cookie 16293682502694286140 [ 4387.887688] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4387.888945] LustreError: 127544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4388.290038] Lustre: server umount lustre-MDT0001 complete [ 4400.329620] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 04:40:09 (1785314409) [ 4410.157079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4410.724928] LustreError: 131124:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4410.783617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4415.980359] LustreError: 131125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4416.176769] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4420.088700] LustreError: 131124:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4425.186886] LustreError: 131125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4429.523306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4438.947310] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4454.546836] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4462.555798] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4474.548228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4475.166662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4475.172492] Lustre: Skipped 1 previous similar message [ 4482.770686] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4492.912816] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4507.794366] Lustre: *** cfs_fail_loc=159, val=0*** [ 4507.815647] Lustre: Skipped 2 previous similar messages [ 4523.285075] Lustre: lustre-MDT0000: Client 4314a448-dc50-45df-b972-93715ffcef13 (at 192.168.204.31@tcp) reconnecting [ 4525.585800] Lustre: *** cfs_fail_loc=159, val=0*** [ 4525.591838] Lustre: Skipped 3 previous similar messages [ 4541.187387] Lustre: lustre-MDT0000: Client 4314a448-dc50-45df-b972-93715ffcef13 (at 192.168.204.31@tcp) reconnecting [ 4562.349767] Lustre: server umount lustre-OST0000 complete [ 4571.629449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4571.643420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4571.667176] Lustre: Skipped 2 previous similar messages [ 4571.686300] Lustre: Skipped 3 previous similar messages [ 4576.745906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4576.756884] Lustre: Skipped 1 previous similar message [ 4578.958341] Lustre: server umount lustre-MDT0000 complete [ 4581.861587] LustreError: 131125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4581.895467] LustreError: 131125:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4582.880858] Lustre: 115291:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785314578/real 1785314578] req@00000000455b6333 x1872037296603776/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1785314594 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4582.880957] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4582.919935] Lustre: 115291:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 4584.585112] LustreError: 131106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314596 with bad export cookie 16293682502694287022 [ 4584.589490] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4584.594932] LustreError: 131106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4585.360842] Lustre: server umount lustre-MDT0001 complete [ 4603.351753] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4607.122694] Key type lgssc unregistered [ 4607.555036] LNet: 134949:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4607.563965] LNetError: 134949:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4607.604834] LNet: Removed LNI 192.168.204.131@tcp [ 4608.971447] Key type .llcrypt unregistered [ 4608.973201] Key type ._llcrypt unregistered [ 4645.151831] Key type ._llcrypt registered [ 4645.156282] Key type .llcrypt registered [ 4645.292427] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4648.096341] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4648.175258] alg: No test for adler32 (adler32-zlib) [ 4649.482334] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4649.925697] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4651.752127] Key type lgssc registered [ 4653.358539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4665.743796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4665.756090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4667.319689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4674.174779] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4677.607611] LustreError: 136366:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4677.629267] LustreError: 136366:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4682.725173] LustreError: 136365:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4687.842472] LustreError: 136366:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4687.941273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4688.840645] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4698.971724] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4709.003470] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4716.827729] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4726.140612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4726.814733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4735.361695] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4737.084320] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4747.227622] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4758.286434] Lustre: *** cfs_fail_loc=159, val=0*** [ 4774.661542] Lustre: lustre-MDT0000: Client 3c3236db-4111-4392-92f5-839393169d52 (at 192.168.204.31@tcp) reconnecting [ 4777.560897] Lustre: *** cfs_fail_loc=159, val=0*** [ 4793.076797] Lustre: lustre-MDT0001: Client 3c3236db-4111-4392-92f5-839393169d52 (at 192.168.204.31@tcp) reconnecting [ 4795.758512] Lustre: *** cfs_fail_loc=159, val=0*** [ 4795.767422] Lustre: Skipped 3 previous similar messages [ 4812.107704] Lustre: lustre-MDT0001: Client 3c3236db-4111-4392-92f5-839393169d52 (at 192.168.204.31@tcp) reconnecting [ 4820.413382] Lustre: 139402:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4823.758472] Lustre: *** cfs_fail_loc=159, val=0*** [ 4823.762713] Lustre: Skipped 4 previous similar messages [ 4839.186269] Lustre: lustre-MDT0000: Client 5e429200-62dc-4c3e-86b9-f5b0c3ef98e4 (at 192.168.204.31@tcp) reconnecting [ 4841.723748] Lustre: *** cfs_fail_loc=159, val=0*** [ 4841.733028] Lustre: Skipped 14 previous similar messages [ 4857.096722] Lustre: lustre-MDT0000: Client 5e429200-62dc-4c3e-86b9-f5b0c3ef98e4 (at 192.168.204.31@tcp) reconnecting [ 4859.307832] Lustre: 139697:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4867.040512] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4867.052755] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4867.074793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4870.115405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4870.125282] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4870.132023] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4870.142769] LustreError: Skipped 1 previous similar message [ 4872.598899] Lustre: server umount lustre-OST0000 complete [ 4878.309826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4878.316417] LustreError: Skipped 2 previous similar messages [ 4878.322423] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4878.332720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4878.335552] Lustre: Skipped 1 previous similar message [ 4880.354227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4880.361785] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4883.617937] Lustre: server umount lustre-MDT0000 complete [ 4885.472386] LustreError: 139465:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4885.490517] LustreError: 139465:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4889.193349] LustreError: 136346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785314901 with bad export cookie 18238769550316043368 [ 4889.207570] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4889.215889] LustreError: 136346:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4890.215485] Lustre: server umount lustre-MDT0001 complete [ 4914.741510] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 4919.091978] Key type lgssc unregistered [ 4919.454580] LNet: 140773:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4919.464066] LNetError: 140773:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4919.485574] LNet: Removed LNI 192.168.204.131@tcp [ 4921.014647] Key type .llcrypt unregistered [ 4921.017165] Key type ._llcrypt unregistered [ 4960.258146] Key type ._llcrypt registered [ 4960.260212] Key type .llcrypt registered [ 4960.331276] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 4961.233942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4961.274311] alg: No test for adler32 (adler32-zlib) [ 4962.548912] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 4963.023389] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 4964.831392] Key type lgssc registered [ 4967.252102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4981.595986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4981.625384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4983.087576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4990.372266] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 4993.512422] LustreError: 142187:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4993.535845] LustreError: 142187:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4998.633189] LustreError: 142188:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5003.579843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5003.747663] LustreError: 142187:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5004.342761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5011.148433] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5024.896818] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5033.355146] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5042.530947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5043.267586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5049.415040] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5053.520843] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5053.559304] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5063.150430] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5071.394979] Lustre: 144774:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5076.599937] Lustre: 144822:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5078.418785] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5079.393223] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5084.609249] Lustre: 144969:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5085.758798] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5087.045805] Lustre: 145068:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5094.370473] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5094.373378] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5094.375237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5094.387200] Lustre: Skipped 1 previous similar message [ 5099.409831] Lustre: server umount lustre-OST0000 complete [ 5104.608923] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5104.609635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5104.632337] Lustre: Skipped 1 previous similar message [ 5104.644704] Lustre: Skipped 2 previous similar messages [ 5109.835227] LustreError: 142187:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5110.013803] Lustre: server umount lustre-MDT0000 complete [ 5114.864575] LustreError: 142184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5114.900764] LustreError: 142184:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5115.633807] LustreError: 144871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785315127 with bad export cookie 17225690225502819238 [ 5115.644021] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5115.645259] LustreError: 144871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5116.321959] Lustre: server umount lustre-MDT0001 complete [ 5134.798552] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5138.959367] Key type lgssc unregistered [ 5139.344566] LNet: 146145:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5139.362180] LNetError: 146145:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5139.388740] LNet: Removed LNI 192.168.204.131@tcp [ 5141.176140] Key type .llcrypt unregistered [ 5141.178109] Key type ._llcrypt unregistered [ 5182.395790] Key type ._llcrypt registered [ 5182.399294] Key type .llcrypt registered [ 5182.520699] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5184.531992] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5184.611796] alg: No test for adler32 (adler32-zlib) [ 5185.964270] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5186.394446] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5188.095192] Key type lgssc registered [ 5189.790293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5200.239515] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5200.291334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5202.174922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5208.934253] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5212.645820] LustreError: 147563:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5212.686064] LustreError: 147563:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5217.764797] LustreError: 147564:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5221.895906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5222.240831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5229.584432] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5239.782854] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5248.192445] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5257.947515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5258.659354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5267.091351] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5267.841724] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5272.968876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5276.211958] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5283.901444] Lustre: *** cfs_fail_loc=159, val=0*** [ 5299.598584] Lustre: lustre-MDT0000: Client b833fd02-ad19-4408-81a1-8fb69046e1aa (at 192.168.204.31@tcp) reconnecting [ 5308.903687] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5308.913768] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5308.916647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5308.943633] Lustre: Skipped 1 previous similar message [ 5313.685807] Lustre: server umount lustre-OST0000 complete [ 5324.266083] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5324.267715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5324.294299] Lustre: Skipped 2 previous similar messages [ 5324.317282] Lustre: Skipped 1 previous similar message [ 5326.514613] Lustre: server umount lustre-MDT0000 complete [ 5329.387549] LustreError: 150068:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5329.412657] LustreError: 150068:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5331.569504] LustreError: 147545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785315343 with bad export cookie 6137010123349762044 [ 5331.578139] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5331.590806] LustreError: 147545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5332.023646] Lustre: server umount lustre-MDT0001 complete [ 5347.024727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5351.525615] Key type lgssc unregistered [ 5352.142614] LNet: 151234:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5352.155396] LNetError: 151234:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5352.186580] LNet: Removed LNI 192.168.204.131@tcp [ 5353.688763] Key type .llcrypt unregistered [ 5353.690154] Key type ._llcrypt unregistered [ 5389.925433] Key type ._llcrypt registered [ 5389.929397] Key type .llcrypt registered [ 5390.166823] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5391.970044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5392.057988] alg: No test for adler32 (adler32-zlib) [ 5393.968700] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5394.442877] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5396.215268] Key type lgssc registered [ 5398.186833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5408.877570] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5408.915922] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5410.301934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5414.896725] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5420.518415] LustreError: 152654:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5420.540693] LustreError: 152654:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5425.647309] LustreError: 152653:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5425.885279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5426.244567] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5432.309923] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5442.981980] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5453.050215] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5467.806821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5468.932596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5472.011424] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5480.247121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5481.059490] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5492.430926] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5516.701437] Lustre: lustre-OST0000: evicting 7fa9a535-1587-4079-940b-49e637c6f50c (at 192.168.204.31@tcp) by administrative request [ 5516.712794] Lustre: Skipped 1 previous similar message [ 5549.027268] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5549.042123] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5549.063737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5552.099166] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5552.107379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5552.114417] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.628432] Lustre: server umount lustre-OST0000 complete [ 5562.338524] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5562.362713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5562.374231] Lustre: Skipped 2 previous similar messages [ 5567.458163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5567.481819] Lustre: Skipped 1 previous similar message [ 5568.590908] Lustre: server umount lustre-MDT0000 complete [ 5572.597047] LustreError: 154096:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5572.655052] LustreError: 154096:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5576.174491] LustreError: 152635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785315588 with bad export cookie 10071643236149083920 [ 5576.183505] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5576.188368] LustreError: 152635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5577.095086] Lustre: server umount lustre-MDT0001 complete [ 5601.392993] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5606.711500] Key type lgssc unregistered [ 5607.235697] LNet: 156756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5607.255780] LNetError: 156756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5607.308465] LNet: Removed LNI 192.168.204.131@tcp [ 5608.585228] Key type .llcrypt unregistered [ 5608.592711] Key type ._llcrypt unregistered [ 5717.781373] Key type ._llcrypt registered [ 5717.785989] Key type .llcrypt registered [ 5717.912295] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5720.103159] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5720.377342] alg: No test for adler32 (adler32-zlib) [ 5721.600774] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5722.042525] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5723.791413] Key type lgssc registered [ 5725.634581] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5735.952447] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5735.979372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5737.767945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5743.884122] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5747.187790] LustreError: 159263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5747.214320] LustreError: 159263:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5752.289658] LustreError: 159264:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5754.216271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5754.910100] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5761.121634] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5771.163826] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5780.548229] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5789.911211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5790.361788] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5798.967094] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 5799.603524] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5807.799321] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5812.184712] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5828.584515] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5828.631297] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5828.635740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5828.673681] Lustre: Skipped 1 previous similar message [ 5829.590950] Lustre: server umount lustre-OST0000 complete [ 5836.768559] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5836.785534] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5836.807521] Lustre: Skipped 1 previous similar message [ 5836.819983] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5838.816164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5838.817734] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5841.155593] Lustre: server umount lustre-MDT0000 complete [ 5843.949652] LustreError: 159264:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5843.984398] LustreError: 159264:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5846.978504] LustreError: 159243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785315858 with bad export cookie 15354623833760007948 [ 5846.982412] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5846.988848] LustreError: 159243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5847.764434] Lustre: server umount lustre-MDT0001 complete [ 5864.892318] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 5868.811124] Key type lgssc unregistered [ 5869.131697] LNet: 162834:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5869.152134] LNetError: 162834:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5870.182532] LNet: Removed LNI 192.168.204.131@tcp [ 5872.097349] Key type .llcrypt unregistered [ 5872.103430] Key type ._llcrypt unregistered [ 5912.141441] Key type ._llcrypt registered [ 5912.144914] Key type .llcrypt registered [ 5912.239648] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 5913.365132] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5913.657792] alg: No test for adler32 (adler32-zlib) [ 5914.937431] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 5915.354442] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 5917.119259] Key type lgssc registered [ 5918.948180] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5938.530822] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:05:47 (1785315947) [ 5963.373296] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 5978.077207] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 6033.528599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6033.566929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6035.051136] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6035.114934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6035.297243] Lustre: lustre-MDT0000: new disk, initializing [ 6035.392713] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6035.446358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6041.754678] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6056.471677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6056.707079] Lustre: 167108:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6056.817200] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6056.827941] Lustre: Skipped 1 previous similar message [ 6056.928287] Lustre: lustre-MDT0001: new disk, initializing [ 6057.097282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6057.151880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6057.168512] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6064.665337] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6079.896614] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6089.149978] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6097.572510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6098.116866] Lustre: lustre-OST0000: new disk, initializing [ 6098.119287] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6098.143350] Lustre: 169078:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6098.341673] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6105.240906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6105.268123] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6105.480639] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6107.103727] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6122.806130] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6141.410923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6141.430466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6141.458320] Lustre: Skipped 1 previous similar message [ 6145.358456] Lustre: server umount lustre-OST0000 complete [ 6154.223889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6154.233681] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6154.266539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6154.275077] Lustre: Skipped 1 previous similar message [ 6156.780898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6156.782664] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6158.288387] Lustre: server umount lustre-MDT0000 complete [ 6162.886521] LustreError: 167099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785316174 with bad export cookie 8050441910641839083 [ 6162.900945] LustreError: 167099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6162.902703] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6163.368457] Lustre: server umount lustre-MDT0001 complete [ 6179.004084] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 6184.497462] Key type lgssc unregistered [ 6184.976960] LNet: 170986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6184.984652] LNetError: 170986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6185.005332] LNet: Removed LNI 192.168.204.131@tcp [ 6186.425885] Key type .llcrypt unregistered [ 6186.430061] Key type ._llcrypt unregistered [ 6224.176123] Key type ._llcrypt registered [ 6224.184101] Key type .llcrypt registered [ 6224.315289] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 6226.153219] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6226.187398] alg: No test for adler32 (adler32-zlib) [ 6227.418873] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 6227.725251] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 6229.449178] Key type lgssc registered [ 6231.130430] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6239.445284] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6239.465792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6241.251705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6247.171740] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6251.488852] LustreError: 172405:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6251.514941] LustreError: 172405:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6256.620146] LustreError: 172406:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6259.927420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6260.503139] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6268.274787] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6277.358281] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6284.430677] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6294.745330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6295.027450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6303.133246] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 6311.455917] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6313.585350] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6321.868939] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6321.894813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6321.910171] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6321.932987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.967542] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6323.180672] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6323.200889] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6323.224032] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6323.234609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.267631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6324.548361] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6324.588466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6324.603160] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6324.612304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.626029] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6326.244522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.754799] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6327.761373] Lustre: Skipped 1 previous similar message [ 6327.787358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6327.801722] Lustre: Skipped 1 previous similar message [ 6327.810716] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6327.820097] Lustre: Skipped 1 previous similar message [ 6327.825768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.843586] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6327.858206] Lustre: Skipped 1 previous similar message [ 6329.119646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.314911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.361231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.608428] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6332.615664] Lustre: Skipped 3 previous similar messages [ 6332.638826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6332.652487] Lustre: Skipped 3 previous similar messages [ 6332.664848] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6332.671179] Lustre: Skipped 3 previous similar messages [ 6332.676355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.689375] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6332.696228] Lustre: Skipped 3 previous similar messages [ 6333.691348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.949149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.271874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.549347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.957678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.348904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.710769] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6341.719216] Lustre: Skipped 6 previous similar messages [ 6341.745640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6341.760337] Lustre: Skipped 6 previous similar messages [ 6341.771447] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6341.777933] Lustre: Skipped 6 previous similar messages [ 6341.786331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.794915] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6341.806781] Lustre: Skipped 6 previous similar messages [ 6343.103728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.371070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.793090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.871847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.327603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.572700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.165622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.316698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.606838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.877545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.107920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.167574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.194251] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6358.199927] Lustre: Skipped 12 previous similar messages [ 6358.213639] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6358.220605] Lustre: Skipped 12 previous similar messages [ 6358.226290] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6358.233610] Lustre: Skipped 12 previous similar messages [ 6358.237353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.259643] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6358.266986] Lustre: Skipped 12 previous similar messages [ 6359.494425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.607151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.054342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.702227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.023213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.774827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.976911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.106137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.243179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.350448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.248126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.566849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.036247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.583186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.889751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.279247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.770026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.834985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.934891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.079088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.196473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.489167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.307773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.555778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.889227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.095739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.201476] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6391.211463] Lustre: Skipped 26 previous similar messages [ 6391.230227] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6391.258717] Lustre: Skipped 26 previous similar messages [ 6391.272352] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6391.285982] Lustre: Skipped 26 previous similar messages [ 6391.306663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.334868] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6391.339320] Lustre: Skipped 26 previous similar messages [ 6392.558191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.680511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.969714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.127427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.338465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.533264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.746381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.954903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.310262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.705236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.651907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.752744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.838767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.894415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.027116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.443620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.470140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.447681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.568080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.380268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.419670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.503093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.502352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.408792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.455040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.657112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.918106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.973192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.027881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.110866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.379097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.510101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.827341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.992595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.094355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.205762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.514503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.650897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.727302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.671326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.788108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.777474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.747427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.724350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.908413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.057221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.131125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.329838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.386490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.596663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.274309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.804994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.807370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.831498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.046892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.173478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.391820] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6456.407231] Lustre: Skipped 56 previous similar messages [ 6456.445769] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6456.466207] Lustre: Skipped 56 previous similar messages [ 6456.483963] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6456.489446] Lustre: Skipped 56 previous similar messages [ 6456.495595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.517503] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6456.530405] Lustre: Skipped 56 previous similar messages [ 6458.018406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.435875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.852922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.081574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.889958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.377996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.785521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.020450] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6468.031912] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6468.046735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.498964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.677226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.968631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.336789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.766431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.888876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.055712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.195566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.485476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.822768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.027293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.206160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.596933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.907107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.307176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.496363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.832146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.248092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.171094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.741951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.234748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.283193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.253493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.055551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.235396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.831319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.049201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.109504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.375418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.181294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.579245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.991037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.493846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.885333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.221462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.375222] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 6522.390315] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 6522.411152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.876438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.281349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.852090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.144369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.272548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.197933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.413853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.604720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.884118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.080297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.336321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.558274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.641544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.954038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.274169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.534412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.824729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.029158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.589539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.860647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.204039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.609542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.781325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.294967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.804425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.143504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.404595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.739423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.873783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.288517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.256378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.976439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.079554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.324144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.985174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.333315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.075367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.746252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.058173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.238205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.434979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.737291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.913456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.844636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.227512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.626555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.962405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.448606] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6585.451631] Lustre: Skipped 91 previous similar messages [ 6585.475241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6585.485100] Lustre: Skipped 91 previous similar messages [ 6585.490162] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6585.493776] Lustre: Skipped 91 previous similar messages [ 6585.497708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.506809] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6585.516063] Lustre: Skipped 91 previous similar messages [ 6586.981815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.261231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.691522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.865842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.919745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.071591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.072529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.106478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.375066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.814453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.071883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.163646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.242343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.461986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.843652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.116133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.297411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.383718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.000172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.526324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.999848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.340881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.538979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.093040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.183957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.862771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.543965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.253889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.892946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.172252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.382214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.016262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.224184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.586388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.030165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.338774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.526922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.653742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.819622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.111788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.415978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.488179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.685932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.902589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.914748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.973221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.243800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.683619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.275742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.531615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.661402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.783448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.732424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.603513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.474712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.563269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.672852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.034028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.307609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.426792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.576746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.743433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.808070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.974511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.104786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.540571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.937033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.430666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.571869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.712879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.516095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.746291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.028364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.242091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.767162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.120733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.404743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.939351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.670412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.886154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.961319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.171229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.273898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.494018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.704436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.027057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.288239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.569230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.551987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.597569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.873866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.976215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.823250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.769015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.972413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.233668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.229548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.444232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.659299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.954185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.087928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.340419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.524258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.490657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.584439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.783974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.963675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.197557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.151338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.357085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.761439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.225448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.714592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.690039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.839494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.241477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.630841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.837320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.119256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.201057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.151197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.270062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.362463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.467566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.592057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.705468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.956519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.396959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.665507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.869589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.980895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.207434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.350846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.508960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.577880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.781062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.199585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.368522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.240635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.364287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.552628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.909765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.541026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.795580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.031281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.265682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.607821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.921380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.264538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.591030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.731688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.071505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.331393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.703077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.047956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.450498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.961187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.775232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.154055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.627141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.842895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.289790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.614469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.719198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.800747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.250545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.373704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.593888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.711226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.892532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.118569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.154433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.200201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.432626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.629974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.997287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.205420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.354466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.554599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.996348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.465745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.616245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.687924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.836626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.824069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.983535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.931488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.923140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.649459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.541325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.587268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.493133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.620561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.660909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.926696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.227366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.523359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.658311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.664345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.430206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.333084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.144791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.206106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.387440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.398436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.745671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.961055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.187313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.505194] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6841.514188] Lustre: Skipped 208 previous similar messages [ 6841.534919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6841.548860] Lustre: Skipped 208 previous similar messages [ 6841.556663] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6841.563405] Lustre: Skipped 208 previous similar messages [ 6841.574678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.587965] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6841.602772] Lustre: Skipped 208 previous similar messages [ 6842.630953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.634080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.467653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.489640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.544205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.490988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.601693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.832435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.996344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.846807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.733292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.069361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.320670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.283587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.307536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.559920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.815546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.074835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.239219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.351318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.263364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.444463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.706130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.950431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.854665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.958393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.462547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.611830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.845922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.906142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.132160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.022681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.258837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.631918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.045116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.272865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.270966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.069640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.919140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.123458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.096113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.324304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.716887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.408220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.678277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.049683] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6894.061173] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6894.090793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.498804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.766728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.133670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.626130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.445561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.857664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.102404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.461193] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6905.473448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.477909] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6907.129439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.396442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.829296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.694729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.150169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.774119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.135750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.653446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.369635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.339612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.419395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.563102] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6926.598202] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6926.631764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6928.108980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.091845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.541757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.201861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.759728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.951170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.005988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.227346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6938.455754] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6938.470035] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6938.477577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.624571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.827074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6941.837300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6942.979589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.312637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6945.557891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6946.753794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6947.921471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.260619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.768413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.276543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6953.722130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.977128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6956.162483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.393768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.554984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6959.622990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.848824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6962.019878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6963.088752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.276260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.660243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.865280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.096135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6969.553069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6970.998176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.188190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6973.074396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.076855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.279927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6976.417833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.713120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.724679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.305157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6981.204179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.085946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.111894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6984.307250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.108641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.049190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.700791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.561052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.606392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6989.533796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.509865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6991.617359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.776351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.807626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.019615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.166967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.312250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.507644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6999.913984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7001.155244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.438521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.573922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7004.491916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.528830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.755911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.151713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7009.544089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.018381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.431226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7013.892899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.680372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.754839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.827325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7017.918139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.161649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7020.277507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.628257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.961741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7024.540989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.959489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7027.521664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7028.873788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7030.114424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7031.323320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.316459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.339303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.222458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7035.097597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7036.225715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.573658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.290068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7040.828214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7042.313018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7043.378264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7044.412852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7045.556520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.619461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7047.607760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7048.814149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7050.659796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7052.810495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7054.347538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7055.781679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7057.413889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7058.741267] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7058.752235] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7058.765957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.278932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7061.672977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7063.198048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.633330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7066.093160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7067.575644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7069.123636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7070.280855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.427330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.866220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7074.222545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.630704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7077.195088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.221264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.293800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.467944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.761497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.252754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7084.824214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7086.219594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.564417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.855961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.319824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.597533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7092.520252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.845160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7095.170578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.810168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7098.194160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7100.177682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.716377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.657331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7103.952219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7105.188888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.394023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7107.240990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.167680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.212606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7110.103254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7111.078094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.050207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7113.214406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.341310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.354029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7116.403600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7117.614729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.369876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7119.216957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.376624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.554656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7122.477722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.562184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.650238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7125.723226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7126.676383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.656716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.597753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.697205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.599587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.764598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7132.714423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.786891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.863235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7136.096909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.293901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7138.378926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.424063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.649840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7141.818995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.949509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.172381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7145.584761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7146.894238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7148.182170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.435780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7150.536403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.846147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7152.999289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7154.184891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.261730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7156.353822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.398738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7158.520373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7159.751882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7161.196925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.467570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7163.548760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7165.074595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7166.509389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.749031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7168.724477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7169.858847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.338458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.854096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.717074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.376878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.956896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7179.132904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7180.096810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.472804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.960576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.149182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.355248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7186.744267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7187.754963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7189.005048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7189.969183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.069168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7192.129428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.418880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7194.740312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.952058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7197.287075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.245218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7199.461716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.508675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7201.698674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.081817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7204.224131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.596692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.775440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7207.806175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7209.258387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7210.708869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7212.013345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.424687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7214.628514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.829831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.864726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7218.030313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7219.499248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.634080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7221.941309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.205529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7224.442743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.881872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.193755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.439487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7229.729960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.023641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.318985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.508504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.576854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.757199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7237.142110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.379895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.385908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7240.628773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7242.065269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.589272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.868347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.165512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.853463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7249.186643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.530449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7251.677696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7253.110372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7254.626587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7256.486231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.423703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.197790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7261.863503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7263.917840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7265.270862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7267.020040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7268.325810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.755897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7271.929181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7273.978749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7275.859753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.589396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7278.773479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7279.983519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.335352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.339434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7283.554139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.841327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.875831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.268077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.460821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7289.732925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.793593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7291.905773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.925190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7294.258181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7295.529939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7296.696382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.899498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7298.998059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7300.032753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7301.100071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.001021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7303.164154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7304.356984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7305.486831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.813289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.946659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7309.092339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.215950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7311.455532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.647448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.611485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7314.757488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.594386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7316.523096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.534105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.717146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.889966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7321.084803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.048994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.965783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.896437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7325.005970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7325.952460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.100943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.246155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.377854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7330.598505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7331.817401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7332.972478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.147172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7335.335078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.494397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.609314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.818396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.987261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.086878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.367141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7343.685424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.697264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7345.834547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7346.996612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.301817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.606268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7350.763658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7351.946977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7353.087746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.092410] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7354.103874] Lustre: Skipped 411 previous similar messages [ 7354.126736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7354.145733] Lustre: Skipped 411 previous similar messages [ 7354.153744] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7354.160773] Lustre: Skipped 411 previous similar messages [ 7354.164800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.174261] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7354.182182] Lustre: Skipped 411 previous similar messages [ 7355.414212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7356.706229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7358.304262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7359.710614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7361.474731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.881586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.091098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7365.484577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.843069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.243275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7369.649684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.084817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7372.128335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7373.361291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7374.731124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.886496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.320325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.350747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.335256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7380.658732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7382.035883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.255582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7384.692452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.904581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.250196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.689094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.655304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.561318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.379589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.783130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7397.191620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.849630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.315308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7402.132536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7403.950117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.778492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7407.283947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.168209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7411.060731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.813234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7413.674337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7415.064242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.106558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.563453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.008640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7420.164397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.305860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.539721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7423.862965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.934395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.268659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7427.336092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.565067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.695529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7430.784298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.998399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.401995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.373256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.422477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.660986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.785473] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7437.797474] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7437.803846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7438.824526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7440.000768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.100975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.239038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7444.089427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.962347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.824649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7449.509387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7450.895156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7452.233967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.781445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7454.917146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.993179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7457.240820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7458.642949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7459.834890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7461.569116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7463.086873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.460874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7465.890890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.093497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.329259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.599503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7471.139064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.355058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7473.350115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7474.566143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.740668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.579190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7477.488855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7478.394759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.559948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7480.587284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.651360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7482.534431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.698260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.865159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.142876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7487.170184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7488.320194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7489.429479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7490.564502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7491.823414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.905393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.978972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7494.948702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.925947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7496.801089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7497.866403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7498.969952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7500.097150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.442232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.777898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7503.832160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.839389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.132256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.387872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.690869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7509.901873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7511.045702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.430265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.811762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.684407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7515.722387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.167481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7518.524556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7520.072855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.428310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.499738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.792091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7524.781378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7526.363156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.661164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7528.801401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.976197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7531.452162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.780968] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7532.805024] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 7532.815311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.820995] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7532.821022] LustreError: 172425:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 7534.207532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7535.612517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.260193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7538.657520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.052175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7541.178336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7542.279207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7543.508925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.808124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7545.876282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.774535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.754325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7548.713140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.957356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.249356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7552.689427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.090522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7555.312520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.406901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7557.465574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7558.542765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7559.600832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.665268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7561.875105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.815568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7564.082448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7565.264198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.480496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.634781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7569.025639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7570.221676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7571.655256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.888899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7574.573708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.168973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.952467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8099.331221] LustreError: 174229:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8099.342945] LustreError: 172425:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8816.608919] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8816.621063] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.644027] Lustre: Skipped 174 previous similar messages [ 8816.669595] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8818.156546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8818.162335] Lustre: Skipped 1 previous similar message [ 8821.362585] Lustre: server umount lustre-OST0000 complete [ 8830.953672] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8830.994151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8833.508171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8833.522539] Lustre: Skipped 1 previous similar message [ 8836.321277] Lustre: server umount lustre-MDT0000 complete [ 8838.629067] LustreError: 172402:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8838.653969] LustreError: 172402:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8841.840815] LustreError: 172386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785318853 with bad export cookie 3324226655706908023 [ 8841.844517] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8841.861499] LustreError: 172386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8842.708294] Lustre: server umount lustre-MDT0001 complete [ 8860.432785] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 8865.350835] Key type lgssc unregistered [ 8865.799730] LNet: 227734:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8865.822964] LNetError: 227734:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8865.877822] LNet: Removed LNI 192.168.204.131@tcp [ 8867.408816] Key type .llcrypt unregistered [ 8867.410466] Key type ._llcrypt unregistered [ 8897.699158] Key type ._llcrypt registered [ 8897.701255] Key type .llcrypt registered [ 8897.851925] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 8898.847283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8898.899124] alg: No test for adler32 (adler32-zlib) [ 8900.837930] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 8901.388272] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 8903.273582] Key type lgssc registered [ 8905.982982] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8921.455665] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8921.497479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8923.374888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8930.175948] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8933.863417] LustreError: 229150:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8933.889666] LustreError: 229150:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8938.976100] LustreError: 229151:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8939.582668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8940.132735] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8946.227922] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8955.878074] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8962.061582] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8969.918432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8970.394723] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8979.256853] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 8986.208584] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8993.991460] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 9001.853853] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 9004.130581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9004.150481] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 9004.155313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9004.173110] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9004.174105] LustreError: 229170:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 9007.350050] Lustre: lustre-OST0000: Not available for connect from 192.168.204.31@tcp (stopping) [ 9009.657251] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9009.693841] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9009.711286] Lustre: Skipped 1 previous similar message [ 9012.400286] Lustre: lustre-OST0000: Not available for connect from 192.168.204.31@tcp (stopping) [ 9013.139490] Lustre: server umount lustre-OST0000 complete [ 9024.207111] LustreError: 229170:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 9050.601117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9050.956400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9058.805837] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9067.588635] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9069.527090] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 9074.474897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9074.503410] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9074.510502] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9074.526080] Lustre: Skipped 1 previous similar message [ 9076.508592] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9229c97d6000.ost_server_uuid 50 [ 9077.897652] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9229c97d6000.ost_server_uuid in FULL state after 0 sec [ 9090.019580] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9090.029795] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9090.053752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9090.078147] Lustre: Skipped 2 previous similar messages [ 9090.106069] Lustre: Skipped 1 previous similar message [ 9091.906650] Lustre: server umount lustre-OST0000 complete [ 9098.721562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9098.737844] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9098.768936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9104.301215] Lustre: server umount lustre-MDT0000 complete [ 9105.394814] LustreError: 229146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9105.423300] LustreError: 229146:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9109.630427] LustreError: 229132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319121 with bad export cookie 7795101530892386579 [ 9109.647448] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9109.650455] LustreError: 229132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9110.363559] Lustre: server umount lustre-MDT0001 complete [ 9129.198967] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [ 9134.744629] Key type lgssc unregistered [ 9135.578190] LNet: 234377:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9135.590244] LNetError: 234377:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9135.643779] LNet: Removed LNI 192.168.204.131@tcp [ 9137.414538] Key type .llcrypt unregistered [ 9137.416287] Key type ._llcrypt unregistered [ 9181.013769] Key type ._llcrypt registered [ 9181.016543] Key type .llcrypt registered [ 9181.138395] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9182.509243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9182.720818] alg: No test for adler32 (adler32-zlib) [ 9184.034617] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [ 9184.452342] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [ 9186.167170] Key type lgssc registered [ 9187.874526] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9201.312901] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9201.374205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9203.710812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9213.064781] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9213.943031] LustreError: 236111:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9213.966019] LustreError: 236111:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9214.949202] LustreError: 236112:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9220.065652] LustreError: 236111:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9223.690612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9224.182516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9230.228446] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9233.491822] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9241.703152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9242.248330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9249.422984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 9251.187200] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9262.583076] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9262.900815] Lustre: lustre-OST0001: new disk, initializing [ 9262.909585] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9262.914620] Lustre: 238435:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9263.063800] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9271.538312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9271.551379] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9271.691907] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9272.132959] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9285.566652] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9296.185236] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9316.322333] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9316.340491] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9316.391815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9317.869435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9317.878621] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9317.929745] Lustre: Skipped 1 previous similar message [ 9322.248201] Lustre: server umount lustre-MDT0000 complete [ 9322.994288] LustreError: 236106:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9323.024955] LustreError: 236106:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9328.136156] LustreError: 237839:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9328.172903] LustreError: 237839:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9328.789179] LustreError: 236092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319340 with bad export cookie 8147037534381669107 [ 9328.797658] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9328.804693] LustreError: 236092:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9329.426940] Lustre: server umount lustre-MDT0001 complete [ 9346.055716] Lustre: server umount lustre-OST0000 complete [ 9349.600561] Lustre: 235372:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785319345/real 1785319345] req@ffff9e23c7c86a00 x1872042861367424/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785319361 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9349.636218] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9349.651431] Lustre: Skipped 1 previous similar message [ 9352.300211] Lustre: server umount lustre-OST0001 complete [ 9383.740488] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9398.886474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9399.253182] LustreError: 241544:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9399.352876] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9404.365691] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9416.555225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9417.153371] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9422.990976] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9426.743916] Lustre: 242686:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9436.511382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9437.206937] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9442.339849] LustreError: 243227:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9442.375649] LustreError: 243227:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9444.411328] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 9448.286957] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9461.306051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9466.896426] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9473.539241] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9483.213294] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9489.673753] Lustre: 244561:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9504.225562] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9504.237698] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9504.253776] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9504.260627] Lustre: Skipped 3 previous similar messages [ 9507.819404] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9507.842433] Lustre: Skipped 2 previous similar messages [ 9510.107736] Lustre: server umount mylustre-MDT0000 complete [ 9512.933698] LustreError: 242278:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9512.963594] LustreError: 242278:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 9515.148705] LustreError: 241519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319527 with bad export cookie 8147037534381671879 [ 9515.149892] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9515.164287] LustreError: 241519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9516.052939] Lustre: server umount mylustre-MDT0001 complete [ 9533.826336] Lustre: server umount mylustre-OST0000 complete [ 9534.240236] Lustre: 235372:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785319530/real 1785319530] req@ffff9e23d1c87800 x1872042861439488/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785319546 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9534.268188] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9534.279684] Lustre: Skipped 3 previous similar messages [ 9536.479771] Lustre: 235374:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785319532/real 1785319532] req@ffff9e24c8d4aa00 x1872042861439744/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785319548 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9539.025451] Lustre: server umount mylustre-OST0001 complete [ 9580.905510] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9598.034975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9598.727438] LustreError: 246893:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9598.878282] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9598.893923] Lustre: Skipped 1 previous similar message [ 9604.308444] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9614.159167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9621.878840] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9625.989908] Lustre: 248035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9636.146208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9644.065364] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 9647.533853] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9660.675329] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9666.564124] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9670.167754] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9682.419132] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9687.491835] Lustre: 249907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9696.741787] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9696.756840] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9696.787410] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 9697.135806] Lustre: server umount tfs-MDT0000 complete [ 9703.269193] LustreError: 249909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319715 with bad export cookie 8147037534381673769 [ 9703.293901] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9703.297611] LustreError: 249909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9704.334160] Lustre: server umount tfs-MDT0001 complete [ 9720.065091] Lustre: server umount tfs-OST0000 complete [ 9723.871139] Lustre: 235372:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785319719/real 1785319719] req@ffff9e24c8880700 x1872042861499520/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785319735 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9723.890721] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9723.900805] Lustre: Skipped 3 previous similar messages [ 9724.630813] Lustre: server umount tfs-OST0001 complete [ 9751.370676] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9768.360029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9769.410644] LustreError: 252156:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9769.437765] LustreError: 252156:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 20 previous similar messages [ 9769.586234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9769.599527] Lustre: Skipped 3 previous similar messages [ 9776.208695] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9788.658387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9795.956299] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9800.061795] Lustre: 253301:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9809.697196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9814.339250] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 9818.548451] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9829.581728] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9835.028967] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9838.033042] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9848.567789] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9853.260804] Lustre: 255174:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9862.468456] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:11:11 (1785319871) [ 9886.183481] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9886.186528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9886.220323] Lustre: Skipped 2 previous similar messages [ 9886.242018] Lustre: Skipped 3 previous similar messages [ 9889.504837] Lustre: server umount lustre-MDT0000 complete [ 9894.671978] LustreError: 252132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785319906 with bad export cookie 8147037534381675435 [ 9894.685742] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9894.717244] LustreError: 252132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9901.373541] Lustre: server umount lustre-MDT0001 complete [ 9907.008595] Lustre: server umount lustre-OST0000 complete [ 9912.625586] Lustre: server umount lustre-OST0001 complete [ 9921.495958] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [ 9937.283321] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [ 9979.439103] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 9979.439103] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9979.439103] [ 9979.465630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9979.822398] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9979.862747] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9979.978788] Lustre: lustre-MDT0000: new disk, initializing [ 9980.118731] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9980.129741] Lustre: Skipped 3 previous similar messages [ 9980.155746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9985.986704] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [ 9999.220166] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 9999.220166] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9999.220166] [ 9999.263769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9999.413155] Lustre: 258383:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9999.500179] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9999.505561] Lustre: Skipped 1 previous similar message [ 9999.621238] Lustre: lustre-MDT0001: new disk, initializing [ 9999.793305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9999.805127] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10005.488462] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10018.535460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10018.942796] Lustre: lustre-OST0000: new disk, initializing [10018.945233] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10018.950755] Lustre: 259964:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10020.410511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10020.422349] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10020.581146] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10027.021175] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10039.368624] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10039.579431] Lustre: lustre-OST0001: new disk, initializing [10039.584674] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10039.591325] Lustre: 260838:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10040.903893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10040.920933] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10041.070969] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10046.705282] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10060.785142] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:14:29 (1785320069) [10119.994216] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:15:28 (1785320128) [10138.597392] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10138.600691] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10138.611503] LustreError: Skipped 1 previous similar message [10138.630573] Lustre: Skipped 3 previous similar messages [10138.637076] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10138.646392] Lustre: Skipped 5 previous similar messages [10143.007171] Lustre: server umount lustre-OST0000 complete [10143.724038] LustreError: 259948:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10143.752599] LustreError: 259948:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [10148.322786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10153.877335] Lustre: server umount lustre-MDT0000 complete [10158.751014] LustreError: 258373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320170 with bad export cookie 8147037534381676989 [10158.764394] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10158.778292] LustreError: 258373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10165.587863] Lustre: server umount lustre-MDT0001 complete [10183.169415] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10202.347211] Lustre: server umount lustre-OST0001 complete [10213.978528] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [10241.085945] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10293.675223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10293.824893] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10293.867410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10293.917463] Lustre: lustre-MDT0000: new disk, initializing [10293.976109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10293.980392] Lustre: Skipped 3 previous similar messages [10294.000453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10300.102224] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10313.697313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10313.921335] Lustre: 265425:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10314.057636] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10314.066211] Lustre: Skipped 1 previous similar message [10314.193192] Lustre: lustre-MDT0001: new disk, initializing [10314.375681] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10314.407180] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10320.171368] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10332.477530] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10340.481430] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10349.711051] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10350.121677] Lustre: lustre-OST0000: new disk, initializing [10350.125606] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10350.145914] Lustre: 267393:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10352.198356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10352.237189] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10352.392760] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10357.906449] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10372.025914] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10383.328414] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10383.338402] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10383.358517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10383.363192] Lustre: Skipped 6 previous similar messages [10383.377357] Lustre: Skipped 6 previous similar messages [10388.694171] Lustre: server umount lustre-OST0000 complete [10400.223437] Lustre: server umount lustre-MDT0000 complete [10405.034778] LustreError: 265415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320417 with bad export cookie 8147037534381678459 [10405.044706] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10405.634526] Lustre: server umount lustre-MDT0001 complete [10418.326144] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10421.477900] Key type lgssc unregistered [10421.780576] LNet: 269155:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10421.785874] LNetError: 269155:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10421.798796] LNet: Removed LNI 192.168.204.131@tcp [10422.624254] Key type .llcrypt unregistered [10422.628643] Key type ._llcrypt unregistered [10461.357953] Key type ._llcrypt registered [10461.359808] Key type .llcrypt registered [10461.522784] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10463.028031] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10463.095938] alg: No test for adler32 (adler32-zlib) [10464.691335] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [10465.274750] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [10467.127205] Key type lgssc registered [10469.153719] Lustre: Echo OBD driver; http://www.lustre.org/ [10484.922052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10484.966049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10487.383971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10496.745968] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10497.509698] LustreError: 270621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10497.532819] LustreError: 270621:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10502.625995] LustreError: 270622:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10507.745467] LustreError: 270621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10508.008406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10508.593770] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10515.043127] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10524.578892] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10533.115938] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10541.975578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10542.586858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10552.636461] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10557.940778] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [10565.632276] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10567.015036] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [10573.280211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10573.285293] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10573.322317] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10573.334566] Lustre: Skipped 1 previous similar message [10576.453852] Lustre: server umount lustre-OST0000 complete [10583.534876] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10583.536984] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10583.559321] Lustre: Skipped 2 previous similar messages [10587.901616] Lustre: server umount lustre-MDT0000 complete [10588.661925] LustreError: 270617:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10588.680084] LustreError: 270617:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10593.464612] LustreError: 272469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320605 with bad export cookie 2595295234529425657 [10593.480592] LustreError: 272469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10593.486768] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10594.667892] Lustre: server umount lustre-MDT0001 complete [10611.598379] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10616.549912] Key type lgssc unregistered [10617.089724] LNet: 274214:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10617.101849] LNetError: 274214:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10617.127536] LNet: Removed LNI 192.168.204.131@tcp [10618.674705] Key type .llcrypt unregistered [10618.678349] Key type ._llcrypt unregistered [10639.631217] Key type ._llcrypt registered [10639.633742] Key type .llcrypt registered [10639.746952] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10640.912627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10641.127047] alg: No test for adler32 (adler32-zlib) [10642.239610] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [10642.552218] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [10644.287160] Key type lgssc registered [10645.343723] Lustre: Echo OBD driver; http://www.lustre.org/ [10655.109470] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10655.144809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10670.879603] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [10676.002044] LustreError: 275411:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.131@tcp: failed processing log, type 4: rc = -110 [10702.751198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10712.602524] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [10722.795477] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10725.671780] Lustre: server umount lustre-OST0000 complete [10751.698871] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [10755.267403] Key type lgssc unregistered [10755.730737] LNet: 277065:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10755.746115] LNetError: 277065:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10755.761841] LNet: Removed LNI 192.168.204.131@tcp [10756.839322] Key type .llcrypt unregistered [10756.841942] Key type ._llcrypt unregistered [10808.424978] Key type ._llcrypt registered [10808.433849] Key type .llcrypt registered [10808.604133] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [10810.539347] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10810.660930] alg: No test for adler32 (adler32-zlib) [10811.903743] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [10812.344994] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [10814.119579] Key type lgssc registered [10815.793837] Lustre: Echo OBD driver; http://www.lustre.org/ [10832.234065] loop0: detected capacity change from 0 to 204800000 [10832.499526] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10832.521862] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10833.217094] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10834.366075] loop0: detected capacity change from 0 to 204800000 [10834.693769] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10834.715959] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10835.484251] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10836.719155] loop0: detected capacity change from 0 to 204800000 [10836.871569] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10836.892074] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10837.081287] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10838.496478] loop0: detected capacity change from 0 to 204800000 [10838.632459] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10838.639468] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10838.736448] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [10840.045303] loop0: detected capacity change from 0 to 204800000 [10840.080719] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10845.169160] loop0: detected capacity change from 0 to 204800000 [10845.206426] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10851.132907] loop0: detected capacity change from 0 to 204800000 [10851.178241] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10856.745387] loop0: detected capacity change from 0 to 204800000 [10856.790239] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [10862.454570] loop0: detected capacity change from 0 to 204800000 [10862.565549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10862.640151] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10862.744368] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10862.754075] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [10862.775761] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [10862.784236] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [10862.792910] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [10862.801722] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [10862.810268] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [10862.824364] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [10862.835949] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [10862.849429] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [10862.863623] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [10862.885507] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [10862.897065] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [10862.907438] Lustre: 280484:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [10870.813557] Lustre: server umount MGS complete [10872.131860] loop0: detected capacity change from 0 to 204800000 [10872.283018] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10873.181741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10873.258244] Lustre: lustre-MDT0000: Aborting client recovery [10874.319101] loop1: detected capacity change from 0 to 204800000 [10874.543965] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10874.676697] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10874.710440] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [10874.748383] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [10874.768941] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [10874.787201] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [10874.812737] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [10874.832434] Lustre: 280917:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [10874.901532] Lustre: Found index 1 for lustre-MDT0001, updating log [10875.295182] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10875.434090] Lustre: lustre-MDT0001: Aborting client recovery [10876.910516] loop2: detected capacity change from 0 to 204800000 [10877.145483] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10877.242080] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10877.269825] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [10877.301850] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10877.316782] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [10877.331622] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10877.346614] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [10877.379404] Lustre: 280997:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10877.704411] Lustre: Found index 0 for lustre-OST0000, updating log [10877.824716] Lustre: 281015:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10877.998807] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10878.087765] Lustre: lustre-OST0000: Aborting recovery [10879.191072] loop3: detected capacity change from 0 to 204800000 [10879.480056] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10879.649074] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10879.674474] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10879.706749] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [10879.738454] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [10879.764852] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10879.794718] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10879.825919] Lustre: 281080:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [10879.934237] Lustre: Found index 1 for lustre-OST0001, updating log [10880.013745] Lustre: 281085:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10880.284323] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10880.311935] LustreError: 280860:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [10880.344502] Lustre: lustre-OST0001: Aborting recovery [10883.308772] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10883.312460] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10883.380759] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [10883.391083] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [10885.347576] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [10885.362503] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [10890.762793] LustreError: 280827:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 562us: rc = -2 [10891.268760] LustreError: 280826:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 590us: rc = -2 [10891.299954] LustreError: 280826:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [10906.085332] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10906.091852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10906.092242] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10906.105113] Lustre: Skipped 3 previous similar messages [10906.133868] Lustre: Skipped 3 previous similar messages [10910.979955] Lustre: server umount lustre-MDT0000 complete [10911.222924] LustreError: 280830:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10911.258124] LustreError: 280830:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [10912.268189] LustreError: 280811:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785320924 with bad export cookie 10485894678191622185 [10912.280505] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10916.332307] LustreError: 280824:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10916.335947] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10916.342713] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10916.366762] LustreError: 280824:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10916.430785] Lustre: Skipped 1 previous similar message [10917.344092] LustreError: 281349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10917.346602] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10917.390173] Lustre: Skipped 2 previous similar messages [10919.219642] Lustre: server umount lustre-MDT0001 complete [10926.857360] Lustre: server umount lustre-OST0000 complete [10934.399233] Lustre: server umount lustre-OST0001 complete [10945.385421] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:29:14 (1785320954) [10982.487150] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [10995.865239] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [11050.981408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11051.248129] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11051.267529] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11051.321771] Lustre: lustre-MDT0000: new disk, initializing [11051.470097] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11051.511708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11057.871089] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11071.690949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11071.915305] Lustre: 285160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11072.038842] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11072.052307] Lustre: Skipped 1 previous similar message [11072.261982] Lustre: lustre-MDT0001: new disk, initializing [11072.436052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11072.497077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11072.518073] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11078.523821] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11089.849931] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11097.181230] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11107.607653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11108.145293] Lustre: lustre-OST0000: new disk, initializing [11108.155609] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11108.167607] Lustre: 287131:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11108.371862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11117.913241] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11118.219381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11118.247084] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11118.526419] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11130.543658] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11139.646870] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11151.733154] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11156.918475] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11200.480204] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11200.503179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11200.553815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11205.608026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11205.624546] Lustre: Skipped 2 previous similar messages [11206.898578] Lustre: server umount lustre-OST0000 complete [11215.841937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11215.845943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11215.864889] Lustre: Skipped 2 previous similar messages [11215.884623] Lustre: Skipped 1 previous similar message [11226.081225] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11226.555069] Lustre: server umount lustre-MDT0000 complete [11230.181566] LustreError: 285167:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11230.227633] LustreError: 285167:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11231.642509] LustreError: 285154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321243 with bad export cookie 10485894678191624908 [11231.648288] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11231.657773] LustreError: 285154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11232.569706] Lustre: server umount lustre-MDT0001 complete [11244.053142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11250.587302] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11255.805179] Lustre: server umount MGS complete [11271.986896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11272.592596] LustreError: 290859:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11272.632663] LustreError: 290872:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11272.676217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11277.767635] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11282.913038] LustreError: 290860:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11282.956622] LustreError: 290860:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11290.948216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11296.950695] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11305.379964] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11313.317391] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11322.945799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11333.587398] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11343.728696] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11378.144225] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11378.167177] LustreError: Skipped 1 previous similar message [11378.169040] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11378.213226] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11378.233154] Lustre: Skipped 4 previous similar messages [11382.244537] Lustre: server umount lustre-OST0000 complete [11388.898255] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11388.918941] LustreError: Skipped 1 previous similar message [11388.927549] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11388.954295] Lustre: Skipped 1 previous similar message [11393.998711] Lustre: server umount lustre-MDT0000 complete [11394.542899] LustreError: 290856:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11394.563451] LustreError: 290856:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11399.301030] LustreError: 290840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321411 with bad export cookie 10485894678191626735 [11399.311327] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11399.324965] LustreError: 290840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11400.154684] Lustre: server umount lustre-MDT0001 complete [11414.286243] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [11418.511451] Key type lgssc unregistered [11418.899895] LNet: 294807:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11418.914411] LNetError: 294807:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11418.934879] LNet: Removed LNI 192.168.204.131@tcp [11419.952747] Key type .llcrypt unregistered [11419.954279] Key type ._llcrypt unregistered [11479.927060] Key type ._llcrypt registered [11479.928711] Key type .llcrypt registered [11480.062578] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [11502.411482] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [11504.606553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11504.685207] alg: No test for adler32 (adler32-zlib) [11506.618898] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [11506.968929] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [11508.727186] Key type lgssc registered [11510.531441] Lustre: Echo OBD driver; http://www.lustre.org/ [11557.380446] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11557.416622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11558.845647] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11558.879921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11558.968681] Lustre: lustre-MDT0000: new disk, initializing [11559.061822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11559.082423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11564.561952] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11578.962532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11579.201356] Lustre: 298773:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11579.363377] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11579.370153] Lustre: Skipped 1 previous similar message [11579.587734] Lustre: lustre-MDT0001: new disk, initializing [11579.756883] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11579.831489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11579.859818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11585.364252] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11597.051811] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11604.491801] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11615.618438] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11616.407747] Lustre: lustre-OST0000: new disk, initializing [11616.433738] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11616.467591] Lustre: 300746:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11616.690114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11624.577120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11624.588135] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11624.785399] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11629.945974] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11643.096222] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11651.131596] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11662.301637] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11669.042133] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11711.969370] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11711.970881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11711.984811] LustreError: Skipped 1 previous similar message [11711.987653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11712.017527] Lustre: Skipped 1 previous similar message [11712.056765] Lustre: Skipped 1 previous similar message [11715.041987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11715.053404] Lustre: Skipped 1 previous similar message [11720.161155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11720.173464] Lustre: Skipped 1 previous similar message [11725.279158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11725.287276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11725.308811] Lustre: Skipped 1 previous similar message [11725.786507] Lustre: server umount lustre-OST0000 complete [11733.474664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11733.486693] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11733.512792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11735.521899] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11737.213867] Lustre: server umount lustre-MDT0000 complete [11742.298797] LustreError: 298763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321754 with bad export cookie 6685520211709541493 [11742.304427] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11742.322045] LustreError: 298763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11742.918595] Lustre: server umount lustre-MDT0001 complete [11755.614138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11762.403951] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11769.845795] Lustre: server umount MGS complete [11784.979176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11785.536436] LustreError: 304569:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11785.564059] LustreError: 304569:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11785.576962] LustreError: 304582:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11785.655262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11790.824245] LustreError: 304570:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11791.664571] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11795.937369] LustreError: 304569:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11801.064536] LustreError: 304570:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11801.132329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11801.744416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11807.083760] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11817.493954] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11826.456448] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11836.019910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11836.530394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11847.850913] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [11860.502832] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11897.314279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11897.351181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11897.366319] Lustre: Skipped 3 previous similar messages [11903.930498] Lustre: server umount lustre-OST0000 complete [11909.088914] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11909.104694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11909.124698] Lustre: Skipped 1 previous similar message [11915.628629] Lustre: server umount lustre-MDT0000 complete [11917.804848] LustreError: 304564:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11917.834906] LustreError: 304564:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11921.529715] LustreError: 304549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785321933 with bad export cookie 6685520211709543320 [11921.551062] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11922.229758] Lustre: server umount lustre-MDT0001 complete [11938.785931] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [11942.926871] Key type lgssc unregistered [11943.374735] LNet: 308521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11943.387350] LNetError: 308521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11943.405946] LNet: Removed LNI 192.168.204.131@tcp [11944.779149] Key type .llcrypt unregistered [11944.782801] Key type ._llcrypt unregistered [11974.147763] Key type ._llcrypt registered [11974.155726] Key type .llcrypt registered [11974.336470] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [11990.704753] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [11992.170243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11992.411617] alg: No test for adler32 (adler32-zlib) [11993.607568] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [11993.984903] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [11995.719159] Key type lgssc registered [11997.039765] Lustre: Echo OBD driver; http://www.lustre.org/ [12026.700337] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [12049.265848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12049.293931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12050.455704] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12050.475552] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12050.529305] Lustre: lustre-MDT0000: new disk, initializing [12050.613649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12050.628516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12056.380539] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12070.737512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12071.155443] Lustre: lustre-OST0000: new disk, initializing [12071.159957] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12071.166831] Lustre: Skipped 1 previous similar message [12071.172450] Lustre: 312591:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12071.314589] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12079.002973] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [12080.313321] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [12080.323736] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [12080.408377] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [13058.536928] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13059.030833] Lustre: server umount lustre-MDT0000 complete [13063.943424] LustreError: 311777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323075 with bad export cookie 4625287678521938950 [13063.950842] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13063.956730] LustreError: 311777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13066.444326] Lustre: server umount lustre-OST0000 complete [13096.897363] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_hostid [13106.505309] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13142.903894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13143.202594] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13143.229262] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13143.344473] Lustre: lustre-MDT0000: new disk, initializing [13143.452337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13143.474182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13148.564770] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13158.936405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13159.080930] Lustre: 316958:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13159.178522] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13159.186666] Lustre: Skipped 1 previous similar message [13159.324225] Lustre: lustre-MDT0001: new disk, initializing [13159.469618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13159.511158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13159.520135] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13164.789566] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13175.167888] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13181.180375] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13188.460496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13188.876775] Lustre: lustre-OST0000: new disk, initializing [13188.890329] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13188.897688] Lustre: 318925:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13189.060926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13195.572694] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13198.430330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13198.443486] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13198.562341] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13206.203925] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13210.981248] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13211.194526] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13215.965610] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13216.203252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13219.299706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13219.300982] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13219.326623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13219.339541] Lustre: Skipped 1 previous similar message [13224.417665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13224.434248] Lustre: Skipped 1 previous similar message [13224.733310] Lustre: server umount lustre-OST0000 complete [13229.536820] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13229.554203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13229.557273] Lustre: Skipped 2 previous similar messages [13229.591596] Lustre: Skipped 1 previous similar message [13234.666520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13234.675131] Lustre: Skipped 1 previous similar message [13234.969735] Lustre: server umount lustre-MDT0000 complete [13239.004217] LustreError: 316949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323250 with bad export cookie 4625287678524879769 [13239.009195] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13239.017904] LustreError: 316949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13239.360752] Lustre: server umount lustre-MDT0001 complete [13242.985880] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [13244.410106] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:07:34 (1785323254) [13250.787588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13251.191714] LustreError: 321237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13251.209732] LustreError: 321237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13251.271356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13255.369466] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13256.676697] LustreError: 321238:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13261.797274] LustreError: 321237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13263.909726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13264.326703] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13269.009416] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13279.580800] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13287.664744] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13296.058847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13296.473138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13302.816708] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13311.309878] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13319.372153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13319.618633] Lustre: lustre-OST0001: new disk, initializing [13319.623882] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [13319.637886] Lustre: 324073:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13319.767566] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13326.872440] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13326.968293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [13326.988985] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [13327.039588] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [13328.100068] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [13328.116302] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [13329.191554] LustreError: 324439:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [13329.211265] LustreError: 324439:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [13331.247254] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [13331.260224] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [13335.412418] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [13335.426769] LustreError: 324433:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [13338.795378] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13339.005332] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13343.659762] LustreError: 324439:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [13343.681572] LustreError: 324439:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [13345.337347] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff922a00f1a000.ost_server_uuid 50 [13347.066354] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff922a00f1a000.ost_server_uuid in FULL state after 0 sec [13353.081285] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff922a00f1a000.ost_server_uuid 50 [13354.959728] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff922a00f1a000.ost_server_uuid in FULL state after 0 sec [13357.195356] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [13372.898340] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13372.917610] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13372.946546] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [13376.942173] Lustre: server umount lustre-OST0001 complete [13378.533603] LustreError: 323059:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13378.559264] LustreError: 323059:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13383.652995] LustreError: 323056:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13383.682030] LustreError: 323056:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13386.281895] Lustre: server umount lustre-OST0000 complete [13392.352244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13392.358406] LustreError: Skipped 2 previous similar messages [13392.363516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13392.380742] Lustre: Skipped 1 previous similar message [13392.390030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13392.392625] Lustre: Skipped 4 previous similar messages [13400.335463] Lustre: server umount lustre-MDT0000 complete [13402.079292] Lustre: 309865:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785323398/real 1785323398] req@00000000724ef674 x1872045806965248/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1785323414 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13402.115149] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13402.135632] Lustre: Skipped 1 previous similar message [13402.152584] LustreError: 323589:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13402.179217] LustreError: 323589:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13404.191145] Lustre: 309865:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785323400/real 1785323400] req@000000000ee844b1 x1872045806966272/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1785323416 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13405.504526] LustreError: 324072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323417 with bad export cookie 4625287678524880651 [13405.512530] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13405.536577] LustreError: 324072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13406.116554] Lustre: server umount lustre-MDT0001 complete [13418.488330] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [13421.803212] Key type lgssc unregistered [13422.111867] LNet: 326761:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13422.118818] LNetError: 326761:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13422.142643] LNet: Removed LNI 192.168.204.131@tcp [13422.993579] Key type .llcrypt unregistered [13422.995471] Key type ._llcrypt unregistered [13446.672617] Key type ._llcrypt registered [13446.676362] Key type .llcrypt registered [13446.797307] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13447.813562] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13447.979676] alg: No test for adler32 (adler32-zlib) [13449.211363] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [13449.560876] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [13451.327324] Key type lgssc registered [13452.840563] Lustre: Echo OBD driver; http://www.lustre.org/ [13461.279829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13461.318712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13462.837729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13468.170436] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13472.232323] LustreError: 328185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13472.248286] LustreError: 328185:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13476.322921] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13477.345381] LustreError: 328184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13482.489797] LustreError: 328185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13483.337639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13484.100985] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13488.678817] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13498.063515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13498.425845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13505.877048] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13507.694403] LustreError: 329863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13507.714325] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [13514.211197] LustreError: 329866:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13514.244249] LustreError: 329866:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13515.429393] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13524.459117] LustreError: 330298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13524.487146] LustreError: 330298:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13525.125125] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13525.522750] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13530.644526] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [13533.031651] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13541.144522] Lustre: DEBUG MARKER: oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13548.258817] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [13548.421637] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13656.547865] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13656.578726] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [13656.588223] Lustre: Skipped 1 previous similar message [13661.664835] LustreError: 329863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13661.677870] LustreError: 329863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13661.740372] Lustre: server umount lustre-OST0001 complete [13667.814498] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13667.822912] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13667.842843] Lustre: Skipped 1 previous similar message [13667.856930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13671.905235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13671.909795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13671.922788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13671.937053] Lustre: Skipped 1 previous similar message [13677.027272] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13677.033650] Lustre: Skipped 1 previous similar message [13682.147908] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13682.152436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13682.170192] Lustre: Skipped 1 previous similar message [13682.495426] Lustre: server umount lustre-OST0000 complete [13687.264472] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13687.291968] Lustre: Skipped 1 previous similar message [13692.387689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13692.403765] Lustre: Skipped 3 previous similar messages [13693.623673] Lustre: server umount lustre-MDT0000 complete [13697.508881] LustreError: 328180:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13697.527651] LustreError: 328180:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [13698.261697] LustreError: 328164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323710 with bad export cookie 10195807466742759259 [13698.268397] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13698.271372] LustreError: 328164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13698.811627] Lustre: server umount lustre-MDT0001 complete [13709.924513] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing unload_modules_local [13712.908429] Key type lgssc unregistered [13713.292197] LNet: 333127:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13713.302856] LNetError: 333127:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13713.324542] LNet: Removed LNI 192.168.204.131@tcp [13714.379946] Key type .llcrypt unregistered [13714.383548] Key type ._llcrypt unregistered [13760.348924] Key type ._llcrypt registered [13760.351331] Key type .llcrypt registered [13760.436710] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13761.621115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13761.646419] alg: No test for adler32 (adler32-zlib) [13762.636532] Lustre: Lustre: Build Version: 2.17.55_22_g38c21b0 [13762.827261] LNet: Added LNI 192.168.204.131@tcp [8/256/0/180] [13764.455321] Key type lgssc registered [13765.589776] Lustre: Echo OBD driver; http://www.lustre.org/ [13775.066149] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13775.094615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13776.543832] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13781.496177] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13786.092551] LustreError: 335420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13786.113195] LustreError: 335420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13789.528937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13789.961545] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13794.043429] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13796.924174] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [13804.080524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13804.565928] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13810.387295] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13814.822516] LustreError: 336911:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13819.238884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13819.579185] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13821.632463] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [13821.648246] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [13826.142158] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13834.407282] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13846.405260] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [13861.859767] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13861.871119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13861.892834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13862.890423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13862.896992] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13862.918449] Lustre: Skipped 2 previous similar messages [13868.005430] LustreError: 337223:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13868.024804] LustreError: 337223:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13868.251236] Lustre: server umount lustre-MDT0000 complete [13872.348174] LustreError: 335401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785323884 with bad export cookie 17235145244221883576 [13872.349070] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13872.354868] LustreError: 335401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13872.837246] Lustre: server umount lustre-MDT0001 complete [13887.483559] Lustre: server umount lustre-OST0000 complete [13889.312332] Lustre: 334683:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785323885/real 1785323885] req@ffff9e24efc20000 x1872047661710848/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785323901 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13889.353561] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13892.010764] Lustre: server umount lustre-OST0001 complete [13909.345293] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing load_modules_local [13919.998470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13920.467590] LustreError: 340685:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13920.564334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13925.149181] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13925.856976] LustreError: 340686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13934.260615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13934.732303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13939.046985] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13942.153698] Lustre: 341825:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [13948.366037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13956.564992] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13959.147779] LustreError: 342180:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13959.165547] LustreError: 342180:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13965.303875] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [13965.986993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13966.310176] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [13966.320728] Lustre: Skipped 1 previous similar message [13971.444548] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [13972.757818] Lustre: DEBUG MARKER: oleg431-server.virtnet: executing set_default_debug -1 all [13981.197934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13985.233885] Lustre: 343695:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [13990.880466] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13990.891562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13990.910232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13990.918882] Lustre: Skipped 3 previous similar messages [13997.048352] LustreError: 340681:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13997.063913] LustreError: 340681:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [13997.182193] Lustre: server umount lustre-MDT0000 complete [14001.124984] LustreError: 340667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785324013 with bad export cookie 17235145244221885277 [14001.149025] LustreError: MGC192.168.204.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14001.153662] LustreError: 340667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14001.635897] Lustre: server umount lustre-MDT0001 complete