[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 512567669 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002321] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004006] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007015] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008009] pid_max: default: 32768 minimum: 301 [ 0.009135] LSM: Security Framework initializing [ 0.010035] Yama: becoming mindful. [ 0.011028] SELinux: Initializing. [ 0.012067] *** VALIDATE selinux *** [ 0.019591] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024513] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025122] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026068] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027099] *** VALIDATE tmpfs *** [ 0.029155] *** VALIDATE proc *** [ 0.030291] *** VALIDATE cgroup *** [ 0.031008] *** VALIDATE cgroup2 *** [ 0.033251] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034153] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036025] Spectre V2 : User space: Vulnerable [ 0.037007] Speculative Store Bypass: Vulnerable [ 0.040482] debug: unmapping init [mem 0xffffffff90e59000-0xffffffff90e60fff] [ 0.043196] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044638] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045011] ... version: 2 [ 0.046008] ... bit width: 48 [ 0.047006] ... generic registers: 4 [ 0.047947] ... value mask: 0000ffffffffffff [ 0.048007] ... max period: 00007fffffffffff [ 0.049007] ... fixed-purpose events: 3 [ 0.049808] ... event mask: 000000070000000f [ 0.051228] rcu: Hierarchical SRCU implementation. [ 0.053524] smp: Bringing up secondary CPUs ... [ 0.054635] x86: Booting SMP configuration: [ 0.055024] .... node #0, CPUs: #1 #2 #3 [ 0.062096] smp: Brought up 1 node, 4 CPUs [ 0.064016] smpboot: Max logical packages: 1 [ 0.065011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.134734] node 0 deferred pages initialised in 67ms [ 0.138200] devtmpfs: initialized [ 0.139225] x86/mm: Memory block size: 128MB [ 0.141951] gcov: version magic: 0x41383552 [ 0.143022] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.144058] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.145229] pinctrl core: initialized pinctrl subsystem [ 0.146138] [ 0.146652] ************************************************************* [ 0.147009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.148007] ** ** [ 0.149008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150007] ** ** [ 0.151010] ** This means that this kernel is built to expose internal ** [ 0.152009] ** IOMMU data structures, which may compromise security on ** [ 0.153008] ** your system. ** [ 0.154007] ** ** [ 0.155006] ** If you see this message and you are not debugging the ** [ 0.156006] ** kernel, report this immediately to your vendor! ** [ 0.157005] ** ** [ 0.158007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159011] ************************************************************* [ 0.160703] NET: Registered protocol family 16 [ 0.161343] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.162030] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.163033] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.164545] cpuidle: using governor menu [ 0.165663] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.167372] PCI: Using configuration type 1 for base access [ 0.169094] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.176048] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.177045] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.179211] cryptd: max_cpu_qlen set to 1000 [ 0.181207] ACPI: Added _OSI(Module Device) [ 0.182007] ACPI: Added _OSI(Processor Device) [ 0.183006] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.184006] ACPI: Added _OSI(Processor Aggregator Device) [ 0.188368] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.195242] ACPI: Interpreter enabled [ 0.196035] ACPI: PM: (supports S0 S3 S4 S5) [ 0.197007] ACPI: Using IOAPIC for interrupt routing [ 0.199085] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.202479] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.211271] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.213031] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.215016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.218070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.221319] acpiphp: Slot [2] registered [ 0.223106] acpiphp: Slot [5] registered [ 0.224616] acpiphp: Slot [6] registered [ 0.225070] acpiphp: Slot [7] registered [ 0.225981] acpiphp: Slot [8] registered [ 0.227341] acpiphp: Slot [9] registered [ 0.229092] acpiphp: Slot [10] registered [ 0.230090] acpiphp: Slot [3] registered [ 0.231059] acpiphp: Slot [4] registered [ 0.232074] acpiphp: Slot [11] registered [ 0.233067] acpiphp: Slot [12] registered [ 0.234126] acpiphp: Slot [13] registered [ 0.236081] acpiphp: Slot [14] registered [ 0.237060] acpiphp: Slot [15] registered [ 0.238020] acpiphp: Slot [16] registered [ 0.238950] acpiphp: Slot [17] registered [ 0.240064] acpiphp: Slot [18] registered [ 0.241067] acpiphp: Slot [19] registered [ 0.242066] acpiphp: Slot [20] registered [ 0.243123] acpiphp: Slot [21] registered [ 0.244049] acpiphp: Slot [22] registered [ 0.245049] acpiphp: Slot [23] registered [ 0.246000] acpiphp: Slot [24] registered [ 0.246000] acpiphp: Slot [25] registered [ 0.247059] acpiphp: Slot [26] registered [ 0.248084] acpiphp: Slot [27] registered [ 0.250078] acpiphp: Slot [28] registered [ 0.251208] acpiphp: Slot [29] registered [ 0.253095] acpiphp: Slot [30] registered [ 0.255075] acpiphp: Slot [31] registered [ 0.256097] PCI host bridge to bus 0000:00 [ 0.258017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.261022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.264019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.267015] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.269017] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.272022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.273184] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.275853] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.278571] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.287913] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.291425] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.293012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.295045] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.296009] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.299797] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.302734] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.305031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.307847] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.313013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.324012] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.330011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.334644] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.342018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.347028] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.367015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.374775] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.382016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.387015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.404021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.422684] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.434016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.441017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.462133] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.477494] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.492016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.503022] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.525018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.544696] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.564038] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.582017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.605018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.623160] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.632558] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.644021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.670014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.695390] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.699433] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.704481] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.707367] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.709260] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.714140] iommu: Default domain type: Passthrough [ 0.718875] SCSI subsystem initialized [ 0.723082] ACPI: bus type USB registered [ 0.726090] usbcore: registered new interface driver usbfs [ 0.728090] usbcore: registered new interface driver hub [ 0.730060] usbcore: registered new device driver usb [ 0.732803] pps_core: LinuxPPS API ver. 1 registered [ 0.735012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.739056] PTP clock support registered [ 0.741050] EDAC MC: Ver: 3.0.0 [ 0.742501] PCI: Using ACPI for IRQ routing [ 0.744159] NetLabel: Initializing [ 0.745007] NetLabel: domain hash size = 128 [ 0.747009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.749083] NetLabel: unlabeled traffic allowed by default [ 0.752443] vgaarb: loaded [ 0.753432] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.755012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.763248] clocksource: Switched to clocksource kvm-clock [ 0.894817] VFS: Disk quotas dquot_6.6.0 [ 0.896294] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.899371] *** VALIDATE ramfs *** [ 0.900280] *** VALIDATE hugetlbfs *** [ 0.902650] pnp: PnP ACPI init [ 0.904247] pnp: PnP ACPI: found 6 devices [ 0.919642] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.922135] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.923441] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.924903] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.926408] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.928300] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.930273] NET: Registered protocol family 2 [ 0.931865] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.935298] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.938916] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.944337] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.947781] TCP: Hash tables configured (established 65536 bind 65536) [ 0.950648] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.954090] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.956992] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.959584] NET: Registered protocol family 1 [ 0.962281] RPC: Registered named UNIX socket transport module. [ 0.964457] RPC: Registered udp transport module. [ 0.966409] RPC: Registered tcp transport module. [ 0.967981] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.970132] NET: Registered protocol family 44 [ 0.971643] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.973450] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.975738] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.978214] PCI: CLS 0 bytes, default 64 [ 0.980142] Unpacking initramfs... [ 2.820728] debug: unmapping init [mem 0xffff9ae2fcc54000-0xffff9ae2fffbffff] [ 2.825489] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.827742] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.830697] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.375417] Initialise system trusted keyrings [ 3.377012] Key type blacklist registered [ 3.378719] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.387504] zbud: loaded [ 3.390234] *** VALIDATE nfs *** [ 3.391570] *** VALIDATE nfs4 *** [ 3.392873] pstore: using deflate compression [ 3.396318] Platform Keyring initialized [ 3.495295] NET: Registered protocol family 38 [ 3.496879] Key type asymmetric registered [ 3.498129] Asymmetric key parser 'x509' registered [ 3.499897] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.502804] io scheduler mq-deadline registered [ 3.504659] io scheduler kyber registered [ 3.506717] io scheduler bfq registered [ 3.508772] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.511910] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.514842] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.518157] ACPI: Power Button [PWRF] [ 3.523316] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.530838] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.544493] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.552708] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.572804] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.602764] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.632039] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.636214] Non-volatile memory driver v1.3 [ 3.637508] Linux agpgart interface v0.103 [ 3.666282] virtio_blk virtio1: [vda] 145976 512-byte logical blocks (74.7 MB/71.3 MiB) [ 3.668511] vda: detected capacity change from 0 to 74739712 [ 3.696556] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.699163] vdb: detected capacity change from 0 to 1073741824 [ 3.722297] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.726168] vdc: detected capacity change from 0 to 2621440000 [ 3.746093] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.750724] vdd: detected capacity change from 0 to 2621440000 [ 3.769266] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.773780] vde: detected capacity change from 0 to 4294967296 [ 3.791884] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.795711] vdf: detected capacity change from 0 to 4294967296 [ 3.804882] libphy: Fixed MDIO Bus: probed [ 3.811848] usbcore: registered new interface driver usbserial_generic [ 3.814240] usbserial: USB Serial support registered for generic [ 3.816395] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.821344] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.822846] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.824486] mousedev: PS/2 mouse device common for all mice [ 3.826248] rtc_cmos 00:05: RTC can wake from S4 [ 3.828755] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.832026] rtc_cmos 00:05: registered as rtc0 [ 3.835721] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.840948] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.842668] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.851469] intel_pstate: CPU model not supported [ 3.854394] hid: raw HID events driver (C) Jiri Kosina [ 3.856417] usbcore: registered new interface driver usbhid [ 3.858340] usbhid: USB HID core driver [ 3.860074] drop_monitor: Initializing network drop monitor service [ 3.862515] Initializing XFRM netlink socket [ 3.864470] NET: Registered protocol family 10 [ 3.867426] Segment Routing with IPv6 [ 3.868952] NET: Registered protocol family 17 [ 3.871156] mpls_gso: MPLS GSO support [ 3.876425] RAS: Correctable Errors collector initialized. [ 3.878428] AVX version of gcm_enc/dec engaged. [ 3.880127] AES CTR mode by8 optimization enabled [ 3.966733] sched_clock: Marking stable (3966707640, 0)->(4826238349, -859530709) [ 3.970547] registered taskstats version 1 [ 3.972468] Loading compiled-in X.509 certificates [ 3.974397] zswap: loaded using pool lzo/zbud [ 4.000867] Key type big_key registered [ 4.013885] Key type encrypted registered [ 4.015793] ima: No TPM chip found, activating TPM-bypass! [ 4.018103] ima: Allocated hash algorithm: sha1 [ 4.019858] ima: No architecture policies found [ 4.022156] evm: Initialising EVM extended attributes: [ 4.024404] evm: security.selinux [ 4.025921] evm: security.ima [ 4.027368] evm: security.capability [ 4.029180] evm: HMAC attrs: 0x1 [ 4.032275] rtc_cmos 00:05: setting system clock to 2026-08-11 20:19:21 UTC (1786479561) [ 4.040741] debug: unmapping init [mem 0xffffffff91e03000-0xffffffff91ffffff] [ 4.044371] debug: unmapping init [mem 0xffffffff90b82000-0xffffffff90e58fff] [ 4.052132] Write protecting the kernel read-only data: 28672k [ 4.056457] debug: unmapping init [mem 0xffffffff8f203000-0xffffffff8f3fffff] [ 4.059748] debug: unmapping init [mem 0xffffffff8fb14000-0xffffffff8fbfffff] [ 4.100272] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.109713] systemd[1]: Detected virtualization kvm. [ 4.111652] systemd[1]: Detected architecture x86-64. [ 4.113670] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.151691] systemd[1]: No hostname configured. [ 4.153361] systemd[1]: Set hostname to . [ 4.155414] random: systemd: uninitialized urandom read (16 bytes read) [ 4.157785] systemd[1]: Initializing machine ID from random generator. [ 4.255314] random: ln: uninitialized urandom read (6 bytes read) [ 4.421986] random: systemd: uninitialized urandom read (16 bytes read) [ 4.424849] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.431338] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.435391] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Journal Service... [ OK ] Reached target Initrd Root Device. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.937482] device-mapper: uevent: version 1.0.3 [ 5.939526] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.870044] random: fast init done [ 6.950017] virtio_net virtio0 ens2: renamed from eth0 [ 7.162758] scsi host0: ata_piix [ 7.197759] scsi host1: ata_piix [ 7.203689] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.210639] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.559064] random: crng init done [ 11.562308] random: 7 urandom warning(s) missed due to ratelimiting [ 14.518985] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 15.590962] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 17.583219] printk: systemd: 26 output lines suppressed due to ratelimiting [ 18.066352] SELinux: Disabled at runtime. [ 18.178459] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.197315] systemd[1]: Detected virtualization kvm. [ 18.200172] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 18.932863] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.937206] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.950138] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.956675] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.962635] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.971776] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.983351] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [ 19.331182] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 19.924172] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.739469] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 20.744901] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.040156] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.067265] EDAC sbridge: Ver: 1.1.2 [ 24.375178] Key type dns_resolver registered [ 24.981194] NFS: Registering the id_resolver key type [ 24.989255] Key type id_resolver registered [ 24.997473] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (6s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg206-server login: [ 52.676647] libcfs: loading out-of-tree module taints kernel. [ 52.696548] Key type ._llcrypt registered [ 52.697699] Key type .llcrypt registered [ 52.745167] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 61.751446] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 62.565873] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 62.582203] alg: No test for adler32 (adler32-zlib) [ 63.668291] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 64.117026] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 65.744500] Key type lgssc registered [ 66.809304] Lustre: Echo OBD driver; http://www.lustre.org/ [ 73.219102] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 90.228803] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 95.384884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 95.393758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 96.507070] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 96.533208] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 96.590680] Lustre: lustre-MDT0000: new disk, initializing [ 96.626502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 96.634207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 98.141741] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 104.014719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 104.059976] Lustre: 6480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 104.082124] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 104.084873] Lustre: Skipped 1 previous similar message [ 104.130532] Lustre: lustre-MDT0001: new disk, initializing [ 104.171425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 104.186364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 104.191212] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 105.863763] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 108.388328] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 112.104583] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 112.200338] Lustre: lustre-OST0000: new disk, initializing [ 112.202311] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 112.204668] Lustre: 8416:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 112.225650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 114.494186] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 118.255994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 118.259795] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 118.292669] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 120.202353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 120.254026] Lustre: lustre-OST0001: new disk, initializing [ 120.256297] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 120.259460] Lustre: 9490:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 120.284091] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 122.619156] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 126.447738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 126.454886] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 126.465632] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 129.884428] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 132.593830] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 134.710914] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing check_logdir /tmp/testlogs/ [ 136.362446] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing yml_node [ 138.130962] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 139.231573] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 140.282299] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 140.917974] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Aug 11 16:21:38 EDT 2026 [ 147.719961] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 148.313384] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 160.736590] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 160.738840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 160.742624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 162.272898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 162.273035] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 162.275776] Lustre: Skipped 2 previous similar messages [ 162.280167] Lustre: Skipped 2 previous similar messages [ 166.308526] Lustre: server umount lustre-MDT0000 complete [ 167.959161] LustreError: 7426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786479725 with bad export cookie 15320162656550907566 [ 167.961128] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 167.963837] LustreError: 7426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 172.513256] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 172.521167] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 172.526219] LustreError: 6485:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 172.526383] Lustre: Skipped 2 previous similar messages [ 172.569632] LustreError: 6485:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 174.264300] Lustre: server umount lustre-MDT0001 complete [ 183.214694] Lustre: server umount lustre-OST0000 complete [ 192.444410] Lustre: server umount lustre-OST0001 complete [ 199.582592] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 210.616587] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 247.330209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 247.503661] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 247.524346] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 247.579234] Lustre: lustre-MDT0000: new disk, initializing [ 247.635951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 247.650512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 251.012824] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 259.001380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 259.109894] Lustre: 15037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 259.148421] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 259.151691] Lustre: Skipped 1 previous similar message [ 259.200299] Lustre: lustre-MDT0001: new disk, initializing [ 259.273756] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 259.303623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 259.320507] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 262.004724] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 268.301771] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 271.832885] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 276.323959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 276.545560] Lustre: lustre-OST0000: new disk, initializing [ 276.548233] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 276.551457] Lustre: 17000:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 276.602564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 278.076644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 278.087172] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 278.185266] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 280.570391] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 286.640674] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 289.697069] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 289.839464] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 292.822797] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 292.962323] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 298.983504] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 298.984990] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 298.985556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 298.993938] Lustre: Skipped 2 previous similar messages [ 298.998396] LustreError: Skipped 1 previous similar message [ 300.332335] Lustre: server umount lustre-OST0000 complete [ 304.097758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 304.110387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 304.116347] Lustre: Skipped 2 previous similar messages [ 314.340522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.350907] Lustre: Skipped 3 previous similar messages [ 317.408150] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 317.601701] Lustre: server umount lustre-MDT0000 complete [ 319.458103] LustreError: 15044:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 319.471452] LustreError: 15044:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 320.637323] LustreError: 15027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786479878 with bad export cookie 15320162656550909099 [ 320.640272] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 320.645995] LustreError: 15027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 320.935799] Lustre: server umount lustre-MDT0001 complete [ 324.725817] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:24:41 (1786479881) [ 329.661492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 330.008120] LustreError: 19268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.068669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 332.961186] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 335.330021] LustreError: 19267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 339.052656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 342.651846] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 347.596942] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 351.078767] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 355.445524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 359.582620] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 364.544817] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 378.318331] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 385.513845] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 385.516750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 385.533815] Lustre: Skipped 3 previous similar messages [ 385.535408] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 385.555424] Lustre: Skipped 2 previous similar messages [ 389.254078] Lustre: server umount lustre-MDT0000 complete [ 390.630909] LustreError: 20074:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 390.652195] LustreError: 20074:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 392.157958] LustreError: 19247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786479949 with bad export cookie 15320162656550909981 [ 392.170848] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 392.327228] Lustre: server umount lustre-MDT0001 complete [ 405.145952] Lustre: server umount lustre-OST0000 complete [ 409.354100] hrtimer: interrupt took 20492724 ns [ 437.141646] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 452.691506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 453.146038] LustreError: 23693:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 453.232812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 453.240823] Lustre: Skipped 2 previous similar messages [ 461.152905] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 473.062311] LustreError: 23694:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 473.087491] LustreError: 23694:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 474.330325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 480.707695] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 484.010798] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 491.579871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 498.163642] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 507.660160] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 507.931569] Lustre: lustre-OST0001: new disk, initializing [ 507.937018] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 507.947945] Lustre: 26007:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 509.928324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 509.946318] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 510.207322] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 518.906353] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 533.209888] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 538.367660] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 551.395173] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 551.401837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 551.418359] Lustre: Skipped 3 previous similar messages [ 551.431420] Lustre: Skipped 3 previous similar messages [ 555.820504] Lustre: server umount lustre-MDT0000 complete [ 556.517467] LustreError: 24540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 556.539959] LustreError: 24540:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 560.665074] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480118 with bad export cookie 15320162656550910828 [ 560.666843] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 560.683672] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 561.089661] Lustre: server umount lustre-MDT0001 complete [ 575.662725] Lustre: server umount lustre-OST0000 complete [ 578.530077] Lustre: 3618:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786480119/real 1786480119] req@ffff9ae37f60d500 x1873259655288448/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786480135 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 579.914592] Lustre: server umount lustre-OST0001 complete [ 599.018693] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 610.462939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 610.964219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 610.970428] Lustre: Skipped 3 previous similar messages [ 616.175707] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 621.541703] LustreError: 28975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 621.564517] LustreError: 28975:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 625.991159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 631.268435] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 634.799507] Lustre: 30124:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 642.075094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 650.128150] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 659.026430] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 666.860823] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 674.609906] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 678.129066] Lustre: 32003:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 685.031151] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 685.052218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 685.057702] Lustre: Skipped 4 previous similar messages [ 685.071568] Lustre: Skipped 3 previous similar messages [ 690.420553] Lustre: server umount lustre-MDT0000 complete [ 694.034261] LustreError: 28955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480251 with bad export cookie 15320162656550912116 [ 694.047250] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 694.371481] Lustre: server umount lustre-MDT0001 complete [ 707.933859] Lustre: server umount lustre-OST0000 complete [ 710.625826] Lustre: 3619:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786480252/real 1786480252] req@ffff9ae24367ce00 x1873259655338752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786480268 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 712.050301] Lustre: server umount lustre-OST0001 complete [ 720.791587] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:31:17 (1786480277) [ 751.046357] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 762.075342] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 762.363521] LustreError: 34806:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 762.379560] LustreError: 34806:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 766.814786] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 776.356467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 782.272896] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 785.663619] Lustre: 35953:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 792.717640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 800.432457] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 808.827381] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 815.935356] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 823.911868] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 828.525272] Lustre: 37831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 835.044665] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 835.050610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 835.056481] Lustre: Skipped 4 previous similar messages [ 835.076946] Lustre: Skipped 8 previous similar messages [ 840.547207] Lustre: server umount lustre-MDT0000 complete [ 844.535524] LustreError: 34788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480402 with bad export cookie 15320162656550913222 [ 844.538461] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 844.553338] LustreError: 34788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 845.063806] Lustre: server umount lustre-MDT0001 complete [ 860.125125] Lustre: server umount lustre-OST0000 complete [ 861.474612] Lustre: 3621:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786480402/real 1786480402] req@ffff9ae34769b480 x1873259655388288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786480418 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 864.522248] Lustre: server umount lustre-OST0001 complete [ 875.354696] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:33:51 (1786480431) [ 891.933394] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 903.887281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 904.542821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 904.553403] Lustre: Skipped 7 previous similar messages [ 909.792763] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 919.305191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 925.310355] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 929.131263] Lustre: 41123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 936.480166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 944.297522] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 953.641513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 960.902404] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 970.512255] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 974.249692] Lustre: 43003:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 975.059292] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 975.066283] Lustre: Skipped 1 previous similar message [ 979.429690] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 979.455182] Lustre: Skipped 4 previous similar messages [ 984.205894] Lustre: server umount lustre-MDT0000 complete [ 988.720526] LustreError: 43007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480546 with bad export cookie 15320162656550914328 [ 988.729940] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 988.736900] LustreError: 43007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 995.344608] Lustre: server umount lustre-MDT0001 complete [ 1004.490866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1014.249271] LustreError: 3617:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9ae36bebe680 x1873259655440640/t0(0) o250->MGC192.168.202.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1018.979086] LustreError: 43879:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1018.997163] LustreError: 43879:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 25 previous similar messages [ 1019.783452] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1024.110943] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1028.492508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1033.435159] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1034.232949] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1034.246335] Lustre: Skipped 1 previous similar message [ 1040.635639] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1042.384972] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1048.197937] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1049.762982] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1059.813940] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1061.346430] Lustre: server umount lustre-MDT0000 complete [ 1064.863874] LustreError: 39956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480622 with bad export cookie 15320162656550915462 [ 1064.865142] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1064.878886] LustreError: 39956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1065.092714] Lustre: server umount lustre-MDT0001 complete [ 1071.311738] Lustre: server umount lustre-OST0000 complete [ 1074.642963] Lustre: server umount lustre-OST0001 complete [ 1083.855249] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:37:20 (1786480640) [ 1102.062585] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1112.380928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.196571] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1126.405784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.140133] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1135.148227] Lustre: 48768:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1141.926518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1148.429469] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1157.439205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1164.529881] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1172.366969] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1176.813285] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1189.861527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1189.882937] Lustre: Skipped 24 previous similar messages [ 1191.236237] Lustre: server umount lustre-MDT0000 complete [ 1194.926512] LustreError: 47599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786480752 with bad export cookie 15320162656550916253 [ 1194.930393] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1194.933119] LustreError: 47599:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1195.464197] Lustre: server umount lustre-MDT0001 complete [ 1205.089329] Lustre: server umount lustre-OST0000 complete [ 1208.127901] Lustre: server umount lustre-OST0001 complete [ 1216.999432] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:39:33 (1786480773) [ 1219.899810] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1221.615497] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:39:38 (1786480778) [ 1231.924584] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 1239.331916] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1259.600902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1259.772116] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1259.830342] Lustre: lustre-MDT0000: new disk, initializing [ 1259.901320] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1264.033239] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1272.609643] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1279.402047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1279.665732] Lustre: lustre-OST0000: new disk, initializing [ 1279.671266] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1279.678958] Lustre: Skipped 1 previous similar message [ 1279.683597] Lustre: 54845:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1281.201455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1281.213555] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1281.251692] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1285.825849] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1294.909954] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1296.739098] Lustre: DEBUG MARKER: create test files [ 1322.468149] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1322.468831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1322.481773] Lustre: Skipped 14 previous similar messages [ 1327.449931] Lustre: server umount lustre-OST0000 complete [ 1330.692693] Lustre: server umount lustre-MDT0000 complete [ 1339.676540] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1341.989890] Key type lgssc unregistered [ 1342.227988] LNet: 56480:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1342.237696] LNetError: 56480:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1342.258457] LNet: Removed LNI 192.168.202.106@tcp [ 1343.396268] Key type .llcrypt unregistered [ 1343.402730] Key type ._llcrypt unregistered [ 1366.529763] Key type ._llcrypt registered [ 1366.531707] Key type .llcrypt registered [ 1366.610684] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1367.485318] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1367.678929] alg: No test for adler32 (adler32-zlib) [ 1368.702743] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 1369.003952] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1370.688134] Key type lgssc registered [ 1371.665278] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1378.101103] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1378.126948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1379.448538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1383.428869] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1389.716649] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1395.280430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1395.548074] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1401.205663] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1403.758418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1403.915532] Lustre: 59195:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1403.923293] Lustre: 59195:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1403.929742] Lustre: 59195:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1403.937078] Lustre: 59195:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1403.944162] Lustre: 59195:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1403.950920] Lustre: 59195:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1406.922940] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1411.334647] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1415.644683] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1423.307915] Lustre: server umount lustre-OST0000 complete [ 1435.775076] Lustre: server umount lustre-MDT0000 complete [ 1443.864889] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1445.849523] Key type lgssc unregistered [ 1446.098886] LNet: 60807:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1446.103822] LNetError: 60807:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1446.115598] LNet: Removed LNI 192.168.202.106@tcp [ 1446.750878] Key type .llcrypt unregistered [ 1446.752631] Key type ._llcrypt unregistered [ 1469.243539] Key type ._llcrypt registered [ 1469.247613] Key type .llcrypt registered [ 1469.355911] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1470.051990] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1470.107742] alg: No test for adler32 (adler32-zlib) [ 1471.140631] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 1471.337509] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1472.984813] Key type lgssc registered [ 1473.792748] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1479.633410] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1479.649377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1481.032781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1484.010518] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1489.892472] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1495.053346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1495.366356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1500.186482] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1505.421773] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1505.655286] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1505.689822] Lustre: 63619:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1505.697053] Lustre: 63619:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1505.702547] Lustre: 63619:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1505.707956] Lustre: 63619:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1505.721076] Lustre: 63619:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1505.723681] Lustre: 63619:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1511.053238] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1521.121504] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1521.130418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1523.046979] Lustre: server umount lustre-OST0000 complete [ 1526.247857] Lustre: server umount lustre-MDT0000 complete [ 1534.895592] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 1536.826586] Key type lgssc unregistered [ 1537.033586] LNet: 65180:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1537.038283] LNetError: 65180:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1537.056145] LNet: Removed LNI 192.168.202.106@tcp [ 1537.714833] Key type .llcrypt unregistered [ 1537.716676] Key type ._llcrypt unregistered [ 1550.974140] Key type ._llcrypt registered [ 1550.976225] Key type .llcrypt registered [ 1551.072515] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 1562.524705] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1563.180923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1563.265074] alg: No test for adler32 (adler32-zlib) [ 1564.260465] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 1564.505338] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 1566.184497] Key type lgssc registered [ 1567.062623] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1593.257186] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1593.284139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1594.459803] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1594.501621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1594.591897] Lustre: lustre-MDT0000: new disk, initializing [ 1594.648439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1594.659663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1597.624536] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1606.585594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1606.682130] Lustre: 68310:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1606.727430] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1606.730893] Lustre: Skipped 1 previous similar message [ 1606.788454] Lustre: lustre-MDT0001: new disk, initializing [ 1606.851542] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1606.884088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1606.891653] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1610.358359] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1617.680867] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1621.451114] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1626.418839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1626.653436] Lustre: lustre-OST0000: new disk, initializing [ 1626.656785] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1626.662364] Lustre: 70279:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1626.728883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1631.142350] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1634.866020] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1634.879374] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1634.962911] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1639.196695] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1645.968812] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1646.244625] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1652.122938] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1652.344269] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1655.789070] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1655.791033] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1655.807082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1655.827953] Lustre: Skipped 1 previous similar message [ 1655.846489] Lustre: Skipped 1 previous similar message [ 1660.982481] Lustre: server umount lustre-OST0000 complete [ 1666.018169] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1666.021069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1666.034938] Lustre: Skipped 1 previous similar message [ 1666.046802] Lustre: Skipped 1 previous similar message [ 1671.145020] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1671.165056] Lustre: Skipped 1 previous similar message [ 1671.769906] Lustre: server umount lustre-MDT0000 complete [ 1676.247441] LustreError: 68303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786481233 with bad export cookie 11573097714616078572 [ 1676.255790] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1676.261266] LustreError: 68303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1677.038693] Lustre: server umount lustre-MDT0001 complete [ 1688.395994] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:47:24 (1786481244) [ 1704.669292] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1751.531850] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 1762.287049] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 1797.744271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1798.046143] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1798.092277] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1798.167874] Lustre: lustre-MDT0000: new disk, initializing [ 1798.224596] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1798.256247] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1802.325388] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1813.496192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1813.672327] Lustre: 76327:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1813.741621] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1813.744063] Lustre: Skipped 1 previous similar message [ 1813.803623] Lustre: lustre-MDT0001: new disk, initializing [ 1813.893376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1813.924327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1813.945277] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1818.578603] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1826.438393] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1831.332500] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1837.848499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1838.267820] Lustre: lustre-OST0000: new disk, initializing [ 1838.279121] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1838.299139] Lustre: 78294:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1838.412753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1843.652216] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1845.829963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1845.837618] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1845.966818] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1853.928449] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1858.388854] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1858.554535] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1862.930444] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1863.123502] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1866.722589] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1866.723874] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1866.727525] LustreError: Skipped 2 previous similar messages [ 1866.728590] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1866.754467] Lustre: Skipped 1 previous similar message [ 1871.156919] Lustre: server umount lustre-OST0000 complete [ 1875.431707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1875.443907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.466253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1875.476426] Lustre: Skipped 1 previous similar message [ 1880.914130] Lustre: server umount lustre-MDT0000 complete [ 1884.010549] LustreError: 76319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786481441 with bad export cookie 11573097714616079454 [ 1884.016726] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1884.030681] LustreError: 76319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1884.267597] Lustre: server umount lustre-MDT0001 complete [ 1891.228852] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:50:48 (1786481448) [ 1897.521664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1897.933126] LustreError: 80706:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1897.949203] LustreError: 80706:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1898.012178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1903.076829] LustreError: 80707:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1903.307450] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1908.194644] LustreError: 80706:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1911.155123] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1917.151173] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1924.022485] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1929.217903] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1936.374845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1936.674900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1936.683819] Lustre: Skipped 1 previous similar message [ 1943.318409] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 1950.475357] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1953.647793] Lustre: 83378:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 8b74dbe1-a5f8-4722-a108-9cae4ec82cd3 at adminstrative request [ 1958.371550] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1958.386218] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1967.587879] Lustre: MGS: Client 8b74dbe1-a5f8-4722-a108-9cae4ec82cd3 (at 0@lo) reconnecting [ 1967.606880] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa09be70df69397d0 to 0xa09be70df6939a70 [ 1967.631908] Lustre: MGC192.168.202.106@tcp: Connection restored to 0@lo (at 0@lo) [ 1991.628557] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1991.806698] Lustre: lustre-OST0001: new disk, initializing [ 1991.810970] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1991.819866] Lustre: 83769:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1999.123967] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2001.002611] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2001.012403] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2001.167425] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2008.375281] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2011.618865] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2011.626438] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.630852] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2011.630860] Lustre: Skipped 2 previous similar messages [ 2011.667621] Lustre: Skipped 2 previous similar messages [ 2016.982127] Lustre: server umount lustre-OST0001 complete [ 2021.862788] LustreError: 83349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2021.869125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2021.883283] LustreError: 83349:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2021.904441] Lustre: Skipped 1 previous similar message [ 2026.717356] Lustre: server umount lustre-OST0000 complete [ 2032.117477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2032.130202] Lustre: Skipped 6 previous similar messages [ 2037.202835] Lustre: server umount lustre-MDT0000 complete [ 2037.256117] LustreError: 82597:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2037.282923] LustreError: 82597:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2041.011868] LustreError: 80688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786481598 with bad export cookie 11573097714616081008 [ 2041.013481] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2041.020123] LustreError: 80688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2041.363890] Lustre: server umount lustre-MDT0001 complete [ 2050.825658] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:53:27 (1786481607) [ 2052.221341] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2054.196299] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:53:30 (1786481610) [ 2055.817205] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2057.955978] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:53:34 (1786481614) [ 2060.187523] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2062.366903] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:53:38 (1786481618) [ 2064.253976] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2066.491338] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:53:42 (1786481622) [ 2068.491802] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2075.305706] loop: module loaded [ 2075.317957] loop0: detected capacity change from 0 to 4294967296 [ 2075.399732] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2077.168166] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2080.817413] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2084.895070] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2109.014628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2109.064826] LustreError: 87895:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2109.069374] LustreError: 87895:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2109.078499] LustreError: 87895:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2109.087774] LustreError: 87895:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2127.465601] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2138.486988] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2180.296339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2180.562790] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2180.587803] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2180.686196] Lustre: lustre-MDT0000: new disk, initializing [ 2180.821308] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2180.825751] Lustre: Skipped 1 previous similar message [ 2180.853284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2185.335928] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2198.074258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2198.211709] Lustre: 90760:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2198.561159] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2204.821591] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2215.426431] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2222.161954] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2228.550348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2228.796358] Lustre: 92730:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2234.863701] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2235.957628] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2236.123288] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2243.418898] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2247.739652] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2247.896701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2251.866941] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2252.005822] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2256.360642] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2256.378597] LustreError: Skipped 1 previous similar message [ 2256.384540] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2256.398111] Lustre: Skipped 2 previous similar messages [ 2256.409625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2259.914320] Lustre: server umount lustre-OST0000 complete [ 2265.058193] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2265.064910] LustreError: Skipped 1 previous similar message [ 2269.583907] Lustre: server umount lustre-MDT0000 complete [ 2272.225967] LustreError: 90769:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2272.257953] LustreError: 90769:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2272.343467] LustreError: 90753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786481829 with bad export cookie 11573097714616081624 [ 2272.350490] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2272.350866] LustreError: 90753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2272.597225] Lustre: server umount lustre-MDT0001 complete [ 2279.624181] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:57:16 (1786481836) [ 2286.331948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2291.188254] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2292.193329] LustreError: 95142:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2292.208650] LustreError: 95142:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2298.509743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2302.181600] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2308.830903] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2313.717692] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2319.784551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2320.053378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2320.056586] Lustre: Skipped 4 previous similar messages [ 2325.660752] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2332.497561] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2336.207260] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2336.363797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2339.911504] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2340.101710] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2346.252286] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2346.390381] Lustre: lustre-OST0001: new disk, initializing [ 2346.394509] Lustre: Skipped 2 previous similar messages [ 2346.397713] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2346.400801] Lustre: Skipped 3 previous similar messages [ 2346.405518] Lustre: 98510:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2351.959654] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2356.333445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2356.342889] Lustre: Skipped 2 previous similar messages [ 2356.345375] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2356.424205] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2359.997797] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2364.378423] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2364.544477] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2368.201492] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2368.394567] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2384.854557] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2406.550901] Lustre: Failing over lustre-MDT0000 [ 2406.918775] Lustre: server umount lustre-MDT0000 complete [ 2407.910907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2407.916893] LustreError: 98518:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.920585] Lustre: Skipped 6 previous similar messages [ 2407.938947] LustreError: 98518:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2424.288139] Lustre: 66534:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786481965/real 1786481965] req@000000000daa903a x1873261228191744/t0(0) o400->MGC192.168.202.106@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786481981 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2424.315684] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2428.251350] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2428.253899] LDISKFS-fs (dm-0): recovery complete [ 2428.262615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2434.530278] LustreError: 66533:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000002b49eb31 x1873261228200064/t0(0) o250->MGC192.168.202.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2434.933566] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2437.193842] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2437.194086] Lustre: 95137:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2438.512821] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2440.162778] Lustre: 95142:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2440.180919] Lustre: 95142:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2440.204789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2440.254535] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2440.577990] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2440.925566] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2440.935124] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2441.256147] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2441.266811] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2441.290341] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2441.305924] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2441.945497] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2441.961834] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2442.288543] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2442.300816] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2443.590567] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2443.607416] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2444.237201] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2444.253550] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2444.576124] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2444.587963] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2447.769533] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2447.790492] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2448.437746] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2448.453937] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2448.776179] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2448.789664] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2456.074216] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2456.089900] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2456.740289] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2456.751784] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2457.080243] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2457.088857] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2472.361958] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2472.377384] Lustre: 101356:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2473.014254] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2473.039384] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2473.360140] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2473.367072] LustreError: 101356:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2497.136245] Lustre: 101356:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2497.144133] Lustre: 101356:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2497.149631] Lustre: 101356:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5c25918a-6580-4258-981b-2acee9a0b574@192.168.202.6@tcp [ 2497.155253] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2497.158577] Lustre: 101356:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@000000008569b80c x1873261217517184/t0(8589934775) o101->5c25918a-6580-4258-981b-2acee9a0b574@192.168.202.6@tcp:310/0 lens 592/0 e 0 to 0 dl 1786482065 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2497.170199] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2497.180266] Lustre: lustre-MDT0000: Denying connection for new client 5c25918a-6580-4258-981b-2acee9a0b574 (at 192.168.202.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2497.185139] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2497.217824] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2497.222555] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2497.225012] Lustre: Skipped 2 previous similar messages [ 2497.273942] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2497.274436] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2508.769853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2508.778593] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2508.792677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2508.803909] Lustre: Skipped 4 previous similar messages [ 2514.661125] Lustre: server umount lustre-OST0000 complete [ 2516.971263] LustreError: 98519:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2516.989900] LustreError: 98519:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 35 previous similar messages [ 2523.801966] Lustre: server umount lustre-OST0001 complete [ 2532.787467] Lustre: server umount lustre-MDT0000 complete [ 2535.782312] LustreError: 98509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482093 with bad export cookie 11573097714616132836 [ 2535.785148] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2535.793526] LustreError: 98509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2536.111914] Lustre: server umount lustre-MDT0001 complete [ 2542.603331] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:01:39 (1786482099) [ 2548.114978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2548.118879] Lustre: *** cfs_fail_loc=197, val=0*** [ 2563.680815] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2568.803604] LustreError: 103344:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.106@tcp: failed processing log, type 4: rc = -110 [ 2573.971798] Lustre: *** cfs_fail_loc=197, val=0*** [ 2573.974768] Lustre: Skipped 65 previous similar messages [ 2594.528354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2594.534237] Lustre: Skipped 2 previous similar messages [ 2599.370122] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2605.299934] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2607.235275] Lustre: server umount lustre-OST0000 complete [ 2613.395509] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:02:50 (1786482170) [ 2631.666234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2635.388971] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2643.021964] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2645.942663] LustreError: 104994:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2645.953865] LustreError: 104994:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2645.959086] LustreError: 104994:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 14, retries 0, failed: rc = -5 [ 2646.149614] Lustre: server umount lustre-MDT0000 complete [ 2668.080414] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2674.948534] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2699.274341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2699.414529] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2699.434605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2699.487755] Lustre: lustre-MDT0000: new disk, initializing [ 2699.542963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2702.123295] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2708.810785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2708.899206] Lustre: 109163:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2709.118710] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2711.928431] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2718.172861] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2721.290139] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2725.333626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2725.562479] Lustre: 111131:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2728.937233] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2734.636472] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2734.709450] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2737.833674] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2737.997433] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2740.559538] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2740.663409] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2744.806535] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2744.807154] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2744.810681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2744.810690] Lustre: Skipped 10 previous similar messages [ 2744.811828] LustreError: Skipped 2 previous similar messages [ 2744.819077] Lustre: Skipped 6 previous similar messages [ 2756.064116] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2756.160340] Lustre: server umount lustre-OST0000 complete [ 2765.054775] Lustre: server umount lustre-MDT0000 complete [ 2765.281651] LustreError: 109175:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2765.292360] LustreError: 109175:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 2767.764117] LustreError: 109156:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482325 with bad export cookie 11573097714616143476 [ 2767.768438] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2767.771889] LustreError: 109156:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2768.008269] Lustre: server umount lustre-MDT0001 complete [ 2773.547979] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:05:30 (1786482330) [ 2779.682243] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 2781.500402] Key type lgssc unregistered [ 2781.722193] LNet: 113622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2781.726478] LNetError: 113622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2781.739592] LNet: Removed LNI 192.168.202.106@tcp [ 2782.218178] Key type .llcrypt unregistered [ 2782.219981] Key type ._llcrypt unregistered [ 2791.855104] Key type ._llcrypt registered [ 2791.856352] Key type .llcrypt registered [ 2791.900242] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2801.156527] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2801.918757] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2801.928144] alg: No test for adler32 (adler32-zlib) [ 2802.894290] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 2803.028722] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 2804.672175] Key type lgssc registered [ 2805.426064] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2830.963850] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2830.989772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2832.144255] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2832.165542] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2832.224549] Lustre: lustre-MDT0000: new disk, initializing [ 2832.317054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2832.327268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2834.991115] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2841.640693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2841.783410] Lustre: lustre-OST0000: new disk, initializing [ 2841.785411] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2841.788905] Lustre: Skipped 1 previous similar message [ 2841.793720] Lustre: 117769:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2841.842727] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2842.975797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2842.982891] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2843.002476] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2845.554604] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2852.558734] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2852.641136] Lustre: lustre-OST0001: new disk, initializing [ 2852.642976] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2852.645950] Lustre: 118633:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2852.687241] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2856.755637] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2858.586410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2858.606732] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2858.635352] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2863.585975] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2863.592159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2863.592288] Lustre: Skipped 1 previous similar message [ 2868.779677] Lustre: server umount lustre-MDT0000 complete [ 2872.907651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2875.726128] LustreError: 116956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482433 with bad export cookie 5389943162398246988 [ 2875.730139] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2875.730601] LustreError: 116956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2875.793375] Lustre: server umount lustre-OST0000 complete [ 2877.790599] Lustre: server umount lustre-OST0001 complete [ 2882.630697] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:07:19 (1786482439) [ 2895.840265] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:07:32 (1786482452) [ 2901.964774] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2916.481950] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:07:53 (1786482473) [ 2941.566272] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 2947.548639] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 2970.500291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2970.699289] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2970.721403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2970.781818] Lustre: lustre-MDT0000: new disk, initializing [ 2970.855147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2970.880479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2973.830696] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2982.493769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2982.561386] Lustre: 126384:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2982.592298] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2982.595710] Lustre: Skipped 1 previous similar message [ 2982.650626] Lustre: lustre-MDT0001: new disk, initializing [ 2982.726165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2982.743106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2982.748930] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2985.874168] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 2992.418186] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2996.324361] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3000.296243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3000.455257] Lustre: lustre-OST0000: new disk, initializing [ 3000.458912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3000.462284] Lustre: 128352:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3000.520481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3004.227351] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3009.591553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3009.598887] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3009.697866] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3011.162498] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3014.008655] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3014.173273] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3017.022871] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3017.143796] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3020.257915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3020.270683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3020.272231] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3020.277154] Lustre: Skipped 1 previous similar message [ 3024.539411] Lustre: server umount lustre-OST0000 complete [ 3028.961565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3028.968392] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3028.983548] Lustre: Skipped 1 previous similar message [ 3028.989449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3028.994756] Lustre: Skipped 1 previous similar message [ 3033.288839] Lustre: server umount lustre-MDT0000 complete [ 3035.521328] LustreError: 126378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482592 with bad export cookie 5389943162398248213 [ 3035.522954] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3035.524772] LustreError: 126378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3035.708637] Lustre: server umount lustre-MDT0001 complete [ 3040.321542] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:09:57 (1786482597) [ 3044.425800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3044.686626] LustreError: 130761:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3044.694609] LustreError: 130761:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3044.725684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3047.506161] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3049.956905] LustreError: 130762:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3052.670335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3055.475689] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3060.569536] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3064.840831] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3069.812819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3073.995678] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3079.037533] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3082.010087] Lustre: *** cfs_fail_loc=159, val=0*** [ 3082.013058] Lustre: Skipped 1 previous similar message [ 3098.221628] Lustre: lustre-MDT0000: Client 07cc1627-1afc-44af-a329-eaea4db7e317 (at 192.168.202.6@tcp) reconnecting [ 3099.274370] Lustre: *** cfs_fail_loc=159, val=0*** [ 3099.277471] Lustre: Skipped 4 previous similar messages [ 3115.618332] Lustre: lustre-MDT0000: Client 07cc1627-1afc-44af-a329-eaea4db7e317 (at 192.168.202.6@tcp) reconnecting [ 3119.078510] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3119.080822] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3119.085034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3119.085042] Lustre: Skipped 2 previous similar messages [ 3119.113312] Lustre: Skipped 2 previous similar messages [ 3124.222520] Lustre: server umount lustre-OST0000 complete [ 3129.315643] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3129.324496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3129.329609] Lustre: Skipped 1 previous similar message [ 3132.896339] Lustre: server umount lustre-MDT0000 complete [ 3134.433666] LustreError: 133420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3134.442369] LustreError: 133420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3135.077894] LustreError: 130743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482692 with bad export cookie 5389943162398249095 [ 3135.082510] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3135.082548] LustreError: 130743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3135.275861] Lustre: server umount lustre-MDT0001 complete [ 3141.570460] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3143.237381] Key type lgssc unregistered [ 3143.416600] LNet: 134539:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3143.420764] LNetError: 134539:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3143.443387] LNet: Removed LNI 192.168.202.106@tcp [ 3144.011117] Key type .llcrypt unregistered [ 3144.013239] Key type ._llcrypt unregistered [ 3159.084633] Key type ._llcrypt registered [ 3159.086678] Key type .llcrypt registered [ 3159.146769] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3159.595919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3159.611075] alg: No test for adler32 (adler32-zlib) [ 3160.526928] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3160.691224] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3162.296120] Key type lgssc registered [ 3163.143419] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3167.774196] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3167.793139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3169.080834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3172.048202] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3177.628698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3177.889035] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3180.821248] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3185.494888] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3189.401094] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3193.751850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3193.942243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3194.989287] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3197.687140] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3202.081926] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3210.183396] Lustre: *** cfs_fail_loc=159, val=0*** [ 3226.724059] Lustre: lustre-MDT0000: Client 5bff3bee-05e4-4c26-88d8-70cd8d6f8b41 (at 192.168.202.6@tcp) reconnecting [ 3228.622756] Lustre: *** cfs_fail_loc=159, val=0*** [ 3228.632080] Lustre: Skipped 2 previous similar messages [ 3244.137338] Lustre: lustre-MDT0001: Client 5bff3bee-05e4-4c26-88d8-70cd8d6f8b41 (at 192.168.202.6@tcp) reconnecting [ 3245.212440] Lustre: *** cfs_fail_loc=159, val=0*** [ 3245.214242] Lustre: Skipped 1 previous similar message [ 3261.568875] Lustre: lustre-MDT0001: Client 5bff3bee-05e4-4c26-88d8-70cd8d6f8b41 (at 192.168.202.6@tcp) reconnecting [ 3276.499324] Lustre: 138994:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3278.472304] Lustre: *** cfs_fail_loc=159, val=0*** [ 3278.476024] Lustre: Skipped 3 previous similar messages [ 3294.797338] Lustre: lustre-MDT0000: Client 9ee73bbc-d9f0-49b6-9334-63fa1fece784 (at 192.168.202.6@tcp) reconnecting [ 3295.824977] Lustre: *** cfs_fail_loc=159, val=0*** [ 3295.829042] Lustre: Skipped 14 previous similar messages [ 3311.200811] Lustre: lustre-MDT0000: Client 9ee73bbc-d9f0-49b6-9334-63fa1fece784 (at 192.168.202.6@tcp) reconnecting [ 3312.108587] Lustre: 139283:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3331.558015] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3331.562572] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3331.566253] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3331.571747] Lustre: Skipped 1 previous similar message [ 3331.579710] Lustre: Skipped 1 previous similar message [ 3336.677399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3336.682128] Lustre: Skipped 1 previous similar message [ 3336.877573] Lustre: server umount lustre-OST0000 complete [ 3341.795367] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3341.796138] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3341.798241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3341.801525] Lustre: Skipped 1 previous similar message [ 3346.097944] Lustre: server umount lustre-MDT0000 complete [ 3346.914198] LustreError: 135940:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3346.931514] LustreError: 135940:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3348.494858] LustreError: 135920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786482905 with bad export cookie 9158058204994887589 [ 3348.497878] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3348.500260] LustreError: 135920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3348.713866] Lustre: server umount lustre-MDT0001 complete [ 3355.486338] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3357.503619] Key type lgssc unregistered [ 3357.701768] LNet: 140363:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3357.709374] LNetError: 140363:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3357.726208] LNet: Removed LNI 192.168.202.106@tcp [ 3358.330844] Key type .llcrypt unregistered [ 3358.333743] Key type ._llcrypt unregistered [ 3373.855431] Key type ._llcrypt registered [ 3373.857263] Key type .llcrypt registered [ 3373.935706] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3374.521730] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3374.535545] alg: No test for adler32 (adler32-zlib) [ 3375.525099] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3375.680646] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3377.312187] Key type lgssc registered [ 3378.115328] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3383.076326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3383.085124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3384.328387] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3387.069854] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3392.588150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3392.887881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3395.684551] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3400.598072] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3404.410388] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3408.516674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3408.741937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3409.782210] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3412.509660] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3416.553434] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3417.964415] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3424.592975] Lustre: 144334:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3426.208058] Lustre: 144382:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3427.122274] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3429.276403] Lustre: 144529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3429.829531] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3429.836885] Lustre: Skipped 1 previous similar message [ 3430.365428] Lustre: 144627:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3433.443221] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3433.444160] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3433.448910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3433.477424] Lustre: Skipped 1 previous similar message [ 3438.542416] Lustre: server umount lustre-OST0000 complete [ 3443.681960] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3443.682934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3443.692362] Lustre: Skipped 1 previous similar message [ 3443.698788] Lustre: Skipped 1 previous similar message [ 3447.023985] Lustre: server umount lustre-MDT0000 complete [ 3448.801591] LustreError: 141751:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3448.813852] LustreError: 141751:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3449.028442] LustreError: 141737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786483006 with bad export cookie 10872673508154336941 [ 3449.033230] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3449.035048] LustreError: 141737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3449.188802] Lustre: server umount lustre-MDT0001 complete [ 3455.202446] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3456.933377] Key type lgssc unregistered [ 3457.140446] LNet: 145689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3457.144766] LNetError: 145689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3457.151499] LNet: Removed LNI 192.168.202.106@tcp [ 3457.653222] Key type .llcrypt unregistered [ 3457.655888] Key type ._llcrypt unregistered [ 3473.237690] Key type ._llcrypt registered [ 3473.239883] Key type .llcrypt registered [ 3473.319438] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3473.858154] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3473.934902] alg: No test for adler32 (adler32-zlib) [ 3474.897859] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3475.087153] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3476.713664] Key type lgssc registered [ 3477.520581] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3482.858432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3482.876113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3484.150613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3486.681984] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3492.669410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3492.929487] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3495.597431] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3500.249527] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3503.749969] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3507.917655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3508.211613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3509.228060] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3512.169832] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3516.880941] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3518.442057] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3524.690266] Lustre: *** cfs_fail_loc=159, val=0*** [ 3540.057959] Lustre: lustre-MDT0000: Client 3f4fa4a8-9fcd-44d0-bded-bdaef5d0e1be (at 192.168.202.6@tcp) reconnecting [ 3544.035958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3544.038254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3544.051782] Lustre: Skipped 1 previous similar message [ 3544.064024] Lustre: Skipped 1 previous similar message [ 3548.732451] Lustre: server umount lustre-OST0000 complete [ 3554.273904] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3554.274608] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3554.279897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3554.290111] Lustre: Skipped 1 previous similar message [ 3557.089178] Lustre: server umount lustre-MDT0000 complete [ 3559.393517] LustreError: 147081:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3559.411774] LustreError: 147081:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3559.429809] LustreError: 147062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786483116 with bad export cookie 7117806625429374109 [ 3559.431256] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3559.437164] LustreError: 147062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3559.694808] Lustre: server umount lustre-MDT0001 complete [ 3565.842336] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3567.468899] Key type lgssc unregistered [ 3567.666284] LNet: 150743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3567.674324] LNetError: 150743:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3567.691214] LNet: Removed LNI 192.168.202.106@tcp [ 3568.137315] Key type .llcrypt unregistered [ 3568.138776] Key type ._llcrypt unregistered [ 3581.505206] Key type ._llcrypt registered [ 3581.507346] Key type .llcrypt registered [ 3581.560966] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3582.218871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3582.240876] alg: No test for adler32 (adler32-zlib) [ 3583.191748] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3583.344712] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3584.952206] Key type lgssc registered [ 3585.622470] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3589.944062] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3589.963159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3591.212890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3593.656540] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3598.313382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3598.551217] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3600.666237] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3604.215627] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3606.668628] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3610.386423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3610.587287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3613.462905] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3615.725527] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3616.905313] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3622.898288] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3638.091667] Lustre: lustre-OST0000: evicting df23266e-25fa-4c26-b955-51009f3e1d10 (at 192.168.202.6@tcp) by administrative request [ 3638.095541] Lustre: Skipped 1 previous similar message [ 3663.844732] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3663.850112] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3663.853509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3663.862938] Lustre: Skipped 1 previous similar message [ 3663.871345] Lustre: Skipped 1 previous similar message [ 3668.970437] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3668.972911] Lustre: Skipped 1 previous similar message [ 3669.392697] Lustre: server umount lustre-OST0000 complete [ 3674.082875] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3674.084587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3674.091698] Lustre: Skipped 1 previous similar message [ 3674.097017] Lustre: Skipped 1 previous similar message [ 3677.806759] Lustre: server umount lustre-MDT0000 complete [ 3679.201094] LustreError: 152137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3679.210943] LustreError: 152137:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3679.815570] LustreError: 152117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786483237 with bad export cookie 10352572506546138581 [ 3679.817191] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3679.822187] LustreError: 152117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3680.027839] Lustre: server umount lustre-MDT0001 complete [ 3685.598591] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3687.348534] Key type lgssc unregistered [ 3687.532571] LNet: 156205:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3687.535654] LNetError: 156205:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3688.550846] LNet: Removed LNI 192.168.202.106@tcp [ 3689.045275] Key type .llcrypt unregistered [ 3689.046730] Key type ._llcrypt unregistered [ 3728.514617] Key type ._llcrypt registered [ 3728.516799] Key type .llcrypt registered [ 3728.604805] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3729.279082] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3729.306056] alg: No test for adler32 (adler32-zlib) [ 3730.218775] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3730.359918] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3731.977593] Key type lgssc registered [ 3732.549693] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3736.880269] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3736.891097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3738.155871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3740.155420] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3744.893981] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3745.087500] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3747.041872] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3751.047686] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3754.098940] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3757.620767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3757.835521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.924836] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3762.996614] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3764.208445] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3770.151235] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3775.459619] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3775.463265] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3775.464505] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3775.473353] Lustre: Skipped 1 previous similar message [ 3780.015444] Lustre: server umount lustre-OST0000 complete [ 3785.698174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3785.701631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3785.704276] Lustre: Skipped 1 previous similar message [ 3785.706713] Lustre: Skipped 1 previous similar message [ 3787.927230] Lustre: server umount lustre-MDT0000 complete [ 3789.707212] LustreError: 158649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786483347 with bad export cookie 3726863413574563191 [ 3789.711259] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3789.714635] LustreError: 158649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3789.840161] Lustre: server umount lustre-MDT0001 complete [ 3794.377710] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3795.535938] Key type lgssc unregistered [ 3795.730710] LNet: 162207:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3795.735796] LNetError: 162207:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3796.774714] LNet: Removed LNI 192.168.202.106@tcp [ 3797.166047] Key type .llcrypt unregistered [ 3797.167199] Key type ._llcrypt unregistered [ 3808.972381] Key type ._llcrypt registered [ 3808.974425] Key type .llcrypt registered [ 3809.039853] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3809.523485] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3809.532658] alg: No test for adler32 (adler32-zlib) [ 3810.430872] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3810.546264] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3812.176313] Key type lgssc registered [ 3812.790277] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3820.290836] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:22:57 (1786483377) [ 3827.498029] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 3831.788040] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3848.111940] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3848.120863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3849.220186] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3849.236020] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3849.270994] Lustre: lustre-MDT0000: new disk, initializing [ 3849.299059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3849.309442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3851.170580] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3856.535354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3856.587063] Lustre: 166442:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3856.619045] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3856.622533] Lustre: Skipped 1 previous similar message [ 3856.661687] Lustre: lustre-MDT0001: new disk, initializing [ 3856.710173] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3856.731084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3856.740424] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3858.808892] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3863.289867] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3865.446444] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3868.296611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3868.416555] Lustre: lustre-OST0000: new disk, initializing [ 3868.418502] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3868.421399] Lustre: 168410:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3868.461760] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3870.925415] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3875.533760] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3876.358528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3876.364316] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3876.411400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3881.441836] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3881.443647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3881.446422] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3881.450027] Lustre: Skipped 1 previous similar message [ 3882.465623] Lustre: server umount lustre-OST0000 complete [ 3886.318470] Lustre: server umount lustre-MDT0000 complete [ 3886.562180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3886.567070] Lustre: Skipped 1 previous similar message [ 3887.920090] LustreError: 166434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786483445 with bad export cookie 18216209645444939618 [ 3887.924675] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3887.929624] LustreError: 166434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3888.109223] Lustre: server umount lustre-MDT0001 complete [ 3892.380959] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 3893.549044] Key type lgssc unregistered [ 3893.713584] LNet: 170314:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3893.716970] LNetError: 170314:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3893.726551] LNet: Removed LNI 192.168.202.106@tcp [ 3894.144953] Key type .llcrypt unregistered [ 3894.146280] Key type ._llcrypt unregistered [ 3905.329491] Key type ._llcrypt registered [ 3905.330831] Key type .llcrypt registered [ 3905.380881] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 3905.960521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3905.973903] alg: No test for adler32 (adler32-zlib) [ 3906.875462] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 3906.999083] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 3908.592210] Key type lgssc registered [ 3909.072385] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3912.684952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3912.696195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3913.920461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3916.068626] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3920.516831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3920.711104] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3922.949647] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3927.020627] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3929.692724] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3932.752891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3932.918554] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3935.698299] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 3938.720126] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3939.047031] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3952.322326] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3952.331027] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3952.337529] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3952.342304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.352161] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3952.757280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.189398] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3953.192671] Lustre: Skipped 1 previous similar message [ 3953.202780] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3953.207428] Lustre: Skipped 1 previous similar message [ 3953.209331] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3953.212142] Lustre: Skipped 1 previous similar message [ 3953.214216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.223204] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3953.226619] Lustre: Skipped 1 previous similar message [ 3953.640980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.045595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.418848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3954.421766] Lustre: Skipped 2 previous similar messages [ 3954.428154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3954.433852] Lustre: Skipped 2 previous similar messages [ 3954.435684] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3954.438559] Lustre: Skipped 2 previous similar messages [ 3954.440395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.446157] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3954.448639] Lustre: Skipped 2 previous similar messages [ 3954.799883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.177814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.544193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.883741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.261995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.642810] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3956.648723] Lustre: Skipped 5 previous similar messages [ 3956.655939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3956.661889] Lustre: Skipped 5 previous similar messages [ 3956.664500] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3956.668261] Lustre: Skipped 5 previous similar messages [ 3956.670746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.675727] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3956.678247] Lustre: Skipped 5 previous similar messages [ 3957.042350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.362172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.763657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.090185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.431102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.879196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.292072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.677660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.089285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.475807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.864879] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3960.867510] Lustre: Skipped 10 previous similar messages [ 3960.878753] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3960.884836] Lustre: Skipped 10 previous similar messages [ 3960.886444] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3960.888917] Lustre: Skipped 10 previous similar messages [ 3960.891187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.904816] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3960.908764] Lustre: Skipped 10 previous similar messages [ 3961.325522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.729187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.085183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.470807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.960198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.387442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.864095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.280322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.677831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.072425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.427270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.814094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.231204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.630629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.013445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.404568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.748826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.119576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.515791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.866393] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3968.869191] Lustre: Skipped 19 previous similar messages [ 3968.877395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.293291] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3969.301131] Lustre: Skipped 20 previous similar messages [ 3969.303243] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3969.305955] Lustre: Skipped 20 previous similar messages [ 3969.307708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.310719] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3969.313158] Lustre: Skipped 20 previous similar messages [ 3969.657225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.086025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.462830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.830965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.208358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.676274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.113719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.532724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.975275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.419589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.851561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.262772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.680664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.138654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.526787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.957323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.311967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.707395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.135803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.574770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.963489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.391118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.829958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.292122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.750494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.113869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.508134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.930246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.301044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.653438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.037046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.372465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.736572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.116599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.486986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.817898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.202540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.520601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.833294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.144872] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3985.147357] Lustre: Skipped 40 previous similar messages [ 3985.153407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.489659] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3985.495294] Lustre: Skipped 40 previous similar messages [ 3985.497259] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3985.500102] Lustre: Skipped 40 previous similar messages [ 3985.501977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.506151] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3985.508479] Lustre: Skipped 40 previous similar messages [ 3985.811992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.122114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.421860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.734365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.043088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.385370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.709685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.093406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.436333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.818771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.198518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.568941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.911888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.267645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.604831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.976170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.334037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.681876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.044359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.467938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.863877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.324939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.765529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.126358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.499229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.964714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.340619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.742889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.139869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.488564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.835995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.306372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.734623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.108775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.498032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.917678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.275644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.654545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.025490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.459431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.816243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.169773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.516187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.889526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.251657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.611222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.951925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.313031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.640107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.991132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.349293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.714341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.100023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.492332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.834694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.193973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.551676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.890589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.251775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.689909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.094248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.451197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.832070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.266227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.643234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.107885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.510382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.884564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.257592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.629783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.989359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.413550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.831982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.277395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.736573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.222385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.676462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.044639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.402053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.746356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.114501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.546759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.975197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.430298] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4017.432503] Lustre: Skipped 84 previous similar messages [ 4017.445121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.953218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4017.959633] Lustre: Skipped 84 previous similar messages [ 4017.962881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4017.966403] Lustre: Skipped 84 previous similar messages [ 4017.968861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.976237] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4017.979160] Lustre: Skipped 84 previous similar messages [ 4018.391636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.818274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.291828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.693962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.107376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.472946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.818267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.174906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.538869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.883621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.274889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.676904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.078152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.447703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.821728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.215422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.599490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.953430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.309272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.671144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.031930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.385241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.728333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.069932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.486241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.921665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.369643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.814936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.245560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.646526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.089253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.514139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.949646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.379096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.758518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.221888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.656072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.110776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.517372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.869231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.231509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.640808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.008481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.360719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.717224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.074301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.433841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.783519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.148690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.541109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.942963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.311552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.684642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.039470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.395142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.754085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.114548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.463282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.816497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.194593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.568317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.953990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.329526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.721204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.139064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.605701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.056645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.464294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.869742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.273187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.625147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.009025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.407139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.791940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.211383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.618558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.041556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.410279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.853723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.230353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.630861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.026938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.402544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.748856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.122146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.481104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.824362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.185340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.565149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.983194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.395854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.859275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.285305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.733562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.245389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.694520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.094716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.478406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.828192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.215774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.729447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.190630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.580983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.976073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.367159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.767420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.208280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.602094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.014400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.415293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.794878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.148662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.611460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.015654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.440455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.831893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.193163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.539240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.876861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.678295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.092356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.569171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.023089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.476460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.871132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.270471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.664383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.043128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.422789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.788371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.159652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.585336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.915157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.329783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.680305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.071255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.483950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.891897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.298902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.694166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.102585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.490204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.861984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.218848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.604355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.954619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.327480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.669179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.019276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.443554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.862874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.207401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.552181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.903531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.258685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.612130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.985745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.351667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.753814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.171734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.512711] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4081.515707] Lustre: Skipped 162 previous similar messages [ 4081.524844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.863762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.184640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4082.189406] Lustre: Skipped 163 previous similar messages [ 4082.191268] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4082.194114] Lustre: Skipped 163 previous similar messages [ 4082.195695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.200054] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4082.202807] Lustre: Skipped 163 previous similar messages [ 4082.527029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.882186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.219140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.607702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.030911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.424970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.809254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.146222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.469248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.863716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.261890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.602362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.988040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.375816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.764216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.177877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.549938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.945949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.270637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.608706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.931840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.258305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.590072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.970141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.308815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.623696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.970478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.359986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.666126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.987082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.344719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.760587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.177692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.611590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.023419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.373044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.722449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.107055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.481794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.836668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.198186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.543455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.871261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.217702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.573752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.908526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.272404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.611052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.970856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.313648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.659088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.046764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.408436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.738881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.088360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.417039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.834058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.198190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.544089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.899761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.311678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.676724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.137484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.577986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.978069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.419819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.840646] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4106.845559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.846343] LustreError: 171723:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4107.201340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.561097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.955755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.315052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.645868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.979759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.340451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.678869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.014678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.356938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.682565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.018802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.364747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.689230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.014077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.345324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.689187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.076187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.462265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.832759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.183289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.530359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.864365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.187220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.537945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.861799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.203235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.587266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.972115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.420907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.767025] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4117.772059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.773957] LustreError: 171723:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4118.107202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.450875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.792912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.116885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.459963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.781041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.142289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.488147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.841221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.180819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.506896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.822446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.184528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.556315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.915209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.288038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.624338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.051417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.428359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.815245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.162717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.472600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.787681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.111690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.396935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.699695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.000519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.325877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.640248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.949346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.274679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.610521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.920994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.263727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.576028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.931454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.306972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.674522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.078833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.466835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.821350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.185485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.557151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.906339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.256842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.627945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.963798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.384892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.757398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.119556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.512100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.866081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.205785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.549545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.882862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.215698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.550654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.913131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.249856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.613309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.954041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.274563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.600659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.951366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.290157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.633061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.016329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.346520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.681134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.010718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.358824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.694840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.053632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.400343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.784835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.109028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.418757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.745835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.058213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.373426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.708315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.044259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.421221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.758124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.142496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.484855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.805912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.122316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.467285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.802264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.140351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.472269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.822569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.141035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.501438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.865085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.220191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.544374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.900287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.257400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.601100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.926979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.280602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.593805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.915939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.242038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.609678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.950710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.297168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.626326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.984069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.313894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.641139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.987606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.369419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.713333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.054417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.400802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.741968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.075472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.414750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.759823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.107473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.431890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.762344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.188474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.610125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.008424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.431315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.783346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.130060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.498647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.837674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.172955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.557463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.886947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.249074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.650191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.023681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.464122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.871075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.306737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.687428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.055276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.439850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.859767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.260916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.633799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.069649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.441856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.791673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.134858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.498981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.848873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.181056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.523062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.848576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.225798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.607744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.936303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.351762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.721141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.096754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.437323] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4175.441923] LustreError: 171723:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4175.442321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.783279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.181924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.551206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.924449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.307370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.711352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.137183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.511617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.831461] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4178.835411] LustreError: 171723:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4178.836457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.160912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.481342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.796530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.146551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.481967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.835633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.201852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.534192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.904423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.225982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.551290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.877826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.194458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.532342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.856195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.183801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.493084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.819043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.142150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.482782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.855868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.249149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.581309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.937884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.324231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.714197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.079052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.413523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.768849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.103785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.450076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.805409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.163474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.539207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.912202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.291532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.772481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.130699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.554972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.940202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.330767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.694870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.074533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.436694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.827330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.228723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.660129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.117841] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4196.123660] LustreError: 171723:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4196.124150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.546154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.962968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.372441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.691538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.036139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.426758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.779336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.149754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.523207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.886954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.223644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.545548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.873757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.169553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.464120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.771562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.084265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.389171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.680871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.004187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.401960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.760189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.101564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.429844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.774665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.157556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.502469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.884456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.284317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.662983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.060735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.413719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.731173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.077654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.423279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.802231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.142300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.522620] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4209.524950] Lustre: Skipped 358 previous similar messages [ 4209.530866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.906809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.251847] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4210.256735] Lustre: Skipped 358 previous similar messages [ 4210.259062] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4210.261310] Lustre: Skipped 358 previous similar messages [ 4210.263114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.267474] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4210.270356] Lustre: Skipped 358 previous similar messages [ 4210.599997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.957644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.295021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.692949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.067280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.453817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.885976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.283140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.715936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.123152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.513943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.909979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.284331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.609352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.944891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.271245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.624395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.010385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.338182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.683259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.022921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.418688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.783805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.212060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.601937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.970102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.324749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.716340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.065354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.413459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.747838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.176674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.523810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.885043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.236618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.582679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.954973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.298069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.634887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.111542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.528156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.857655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.237941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.664050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.015638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.385077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.725686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.114573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.443932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.760200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.124137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.469393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.796145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.143656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.499516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.831583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.181919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.554837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.888927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.291574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.697473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.042983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.420039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.785683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.187642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.542762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.926360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.317661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.661079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.989354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.344310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.680256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.995857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.332750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.743282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.091700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.472421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.845566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.240447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.639482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.083771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.515890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.870669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.243896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.622051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.989114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.426586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.833364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.203980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.545247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.894866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.261748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.628323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.031461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.394095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.770111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.148091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.548972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.954870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.350624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.692265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.054722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.373256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.702197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.042698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.390748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.780025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.143473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.522445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.875223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.240930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.582561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.922459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.247710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.568184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.915647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.254703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.586403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.911568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.243278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.572726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.933413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.275169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.664237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.009124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.356620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.700773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.107078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.475445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.822613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.171725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.511474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.835351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.145786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.484850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.861029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.227765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.602337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.950750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.317334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.660605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.017292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.348110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.686460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.048198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.396942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.747567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.112646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.465845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.790475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.126838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.443320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.772604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.138917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.866430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.234587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.672048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.132741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.543619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.992343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.397203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.800166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.174449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.516511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.898804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.279393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.653468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.027348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.361108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.728444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.086196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.451745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.824138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.180111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.557988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.910505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.306057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.757797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.252898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.664787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.017439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.375363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.733330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.058634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.399115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.736197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.051285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.405513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.743477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.066390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.396586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.698575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.056390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.375700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.699657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.042203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.396655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.723859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.070276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.395174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.728967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.078793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.398913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.770373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.213658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.569260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.929328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.295664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.641460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.037135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.398258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.771659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.106593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.473868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.830315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.186356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.591293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.947397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.279696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.649345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.997476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.391375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.762463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.112462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.504255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.872803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.273166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.683904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.064381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.450227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.819295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.224886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.590506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.936566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.349362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.721425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.130347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.551516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.928916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.343133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.779849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.186208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.575537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.970464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.399580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.791667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.156077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.575394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.958177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.360871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.723689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.081362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.517151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.900910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.280790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.762590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.148956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.564914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.975052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.370942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.763145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.135624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.538938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.902818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.286341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.657195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.034164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.464060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.867901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.279365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.676116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.088505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.453165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.835726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.204578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.562462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.899684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.297512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.621231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.987696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.302883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.643490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.017853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.352393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.699413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.024472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.349166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.731674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.073119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.430951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.789445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.186035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.544483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.904090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.339591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.737948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.151925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.583371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.016869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.385347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.738491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.150554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.546696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.008135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.423094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.809489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.248713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.552300] LustreError: 173523:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4691.557586] LustreError: 171723:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5011.427185] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5011.428831] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5011.437111] Lustre: Skipped 307 previous similar messages [ 5011.443384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5016.548770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5016.553262] Lustre: Skipped 2 previous similar messages [ 5017.196357] Lustre: server umount lustre-OST0000 complete [ 5021.668675] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5021.668752] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5021.677516] Lustre: Skipped 1 previous similar message [ 5025.385908] Lustre: server umount lustre-MDT0000 complete [ 5026.785118] LustreError: 218352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5026.790710] LustreError: 218352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5026.960296] LustreError: 171686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484584 with bad export cookie 8455653943389845001 [ 5026.962802] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5026.964996] LustreError: 171686:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5027.129620] Lustre: server umount lustre-MDT0001 complete [ 5031.889996] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5033.083380] Key type lgssc unregistered [ 5033.233686] LNet: 226837:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5033.237343] LNetError: 226837:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5033.246469] LNet: Removed LNI 192.168.202.106@tcp [ 5033.594862] Key type .llcrypt unregistered [ 5033.596645] Key type ._llcrypt unregistered [ 5044.491399] Key type ._llcrypt registered [ 5044.492973] Key type .llcrypt registered [ 5044.539728] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5044.928616] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5044.977400] alg: No test for adler32 (adler32-zlib) [ 5045.859814] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 5045.967254] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5047.552189] Key type lgssc registered [ 5048.063220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5051.619575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5051.625331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.809711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5054.374295] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5057.548992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5057.682436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5059.168367] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5061.973862] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5064.086554] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5066.691197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5066.843176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5069.432613] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5072.533565] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5078.126484] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 5081.121974] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5082.000519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5082.010681] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5082.014791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.022559] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5082.023959] LustreError: 228249:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5083.620247] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5083.623304] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.626764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5083.627706] Lustre: Skipped 1 previous similar message [ 5084.239463] Lustre: lustre-OST0000: Not available for connect from 192.168.202.6@tcp (stopping) [ 5084.242695] Lustre: Skipped 1 previous similar message [ 5085.041490] Lustre: server umount lustre-OST0000 complete [ 5102.120247] LustreError: 228249:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5113.776249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5113.903235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5115.995892] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5118.752436] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5124.130573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.135026] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.135065] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5124.141523] Lustre: Skipped 1 previous similar message [ 5125.427591] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 6 sec [ 5127.328590] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff995edfd16800.ost_server_uuid 50 [ 5127.848672] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff995edfd16800.ost_server_uuid in FULL state after 0 sec [ 5129.186036] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5129.189396] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5129.192180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5129.194242] Lustre: Skipped 1 previous similar message [ 5134.307156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5134.309482] Lustre: Skipped 2 previous similar messages [ 5135.002446] Lustre: server umount lustre-OST0000 complete [ 5139.425478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5139.426334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5139.429014] Lustre: Skipped 1 previous similar message [ 5139.432860] Lustre: Skipped 1 previous similar message [ 5142.558389] Lustre: server umount lustre-MDT0000 complete [ 5143.852975] LustreError: 228211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484701 with bad export cookie 5411311388117143912 [ 5143.855860] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5143.856832] LustreError: 228211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5143.975329] Lustre: server umount lustre-MDT0001 complete [ 5147.446214] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5148.478476] Key type lgssc unregistered [ 5148.602397] LNet: 233437:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5148.606161] LNetError: 233437:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5148.615447] LNet: Removed LNI 192.168.202.106@tcp [ 5148.920810] Key type .llcrypt unregistered [ 5148.922337] Key type ._llcrypt unregistered [ 5159.918685] Key type ._llcrypt registered [ 5159.920096] Key type .llcrypt registered [ 5159.958864] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5160.367571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5160.393808] alg: No test for adler32 (adler32-zlib) [ 5161.245671] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 5161.328963] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5162.904149] Key type lgssc registered [ 5163.262198] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5166.168257] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5166.174668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5167.340037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5168.683449] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5171.436627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5171.556180] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5172.877896] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5173.788826] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5175.939814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5176.065134] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5177.128538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5177.961947] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5180.849413] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5180.914444] Lustre: lustre-OST0001: new disk, initializing [ 5180.916686] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5180.919560] Lustre: 237452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5180.954021] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5182.963790] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5186.055928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5186.058787] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5186.091924] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5188.403071] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5192.134815] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5201.379472] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5201.380838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5201.384017] Lustre: Skipped 3 previous similar messages [ 5201.387237] Lustre: Skipped 3 previous similar messages [ 5203.678300] Lustre: server umount lustre-MDT0000 complete [ 5204.920164] LustreError: 235121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484762 with bad export cookie 17777050397520080019 [ 5204.922132] LustreError: 235879:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 5204.925195] LustreError: 235121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5204.925650] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5205.042516] Lustre: server umount lustre-MDT0001 complete [ 5215.682705] Lustre: server umount lustre-OST0000 complete [ 5227.168742] Lustre: server umount lustre-OST0001 complete [ 5233.927361] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5237.449177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5237.592587] LustreError: 240548:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5237.600819] LustreError: 240548:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5237.618588] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5238.976913] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5241.854954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5243.327883] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5244.309427] Lustre: 241690:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5246.485719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5246.608077] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5246.610966] Lustre: Skipped 1 previous similar message [ 5247.652136] LustreError: 242044:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5247.655477] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 5248.471592] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5251.263159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5252.402029] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 5253.302566] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5256.399240] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5257.625155] Lustre: 243559:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5262.820368] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5262.821486] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5262.821650] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5262.824778] Lustre: Skipped 3 previous similar messages [ 5267.941049] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5267.943299] Lustre: Skipped 6 previous similar messages [ 5268.062111] Lustre: server umount mylustre-MDT0000 complete [ 5269.284059] LustreError: 240524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484826 with bad export cookie 17777050397520082700 [ 5269.287836] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5269.288439] LustreError: 240524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5269.399145] Lustre: server umount mylustre-MDT0001 complete [ 5281.280590] Lustre: server umount mylustre-OST0000 complete [ 5291.747676] Lustre: server umount mylustre-OST0001 complete [ 5298.683190] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5302.470374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5302.647782] LustreError: 245865:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5302.673358] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5302.675529] Lustre: Skipped 1 previous similar message [ 5304.164407] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5307.002760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5308.482779] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5309.387256] Lustre: 247006:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5311.529168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5312.676874] LustreError: 247361:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5312.679914] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 5313.468422] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5316.125529] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5317.227680] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 5318.039862] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5321.114981] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5322.280259] Lustre: 248876:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5327.844179] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5327.845979] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5327.847637] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5327.848781] Lustre: Skipped 3 previous similar messages [ 5327.850570] Lustre: Skipped 2 previous similar messages [ 5330.387548] Lustre: server umount tfs-MDT0000 complete [ 5331.630313] LustreError: 245840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484889 with bad export cookie 17777050397520084653 [ 5331.633927] LustreError: 245840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5331.634066] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5331.753752] Lustre: server umount tfs-MDT0001 complete [ 5343.322625] Lustre: server umount tfs-OST0000 complete [ 5354.942051] Lustre: server umount tfs-OST0001 complete [ 5361.979484] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5365.449272] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5365.599769] LustreError: 251124:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5365.618302] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5365.621043] Lustre: Skipped 3 previous similar messages [ 5366.948428] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5369.733089] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5371.253759] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5372.238333] Lustre: 252265:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5374.516775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5375.651037] LustreError: 252619:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5375.656273] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 5376.602464] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5379.368194] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5380.460323] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 5381.434160] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5384.584070] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5385.779307] Lustre: 254135:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5387.988708] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:49:05 (1786484945) [ 5400.545047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5400.547592] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5400.551931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5400.553751] Lustre: Skipped 1 previous similar message [ 5405.925026] Lustre: server umount lustre-MDT0000 complete [ 5406.177637] LustreError: 251124:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5406.184132] LustreError: 251124:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5407.176438] LustreError: 251099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786484964 with bad export cookie 17777050397520086340 [ 5407.179954] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5407.180820] LustreError: 251099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5411.297870] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5411.299705] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5411.302104] Lustre: Skipped 4 previous similar messages [ 5411.304336] Lustre: Skipped 4 previous similar messages [ 5413.358268] Lustre: server umount lustre-MDT0001 complete [ 5420.672448] Lustre: server umount lustre-OST0000 complete [ 5436.384104] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5436.437834] Lustre: server umount lustre-OST0001 complete [ 5438.307699] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 5441.205946] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5452.831620] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5452.831620] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5452.831620] [ 5452.839793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5452.917368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5452.926782] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5452.962789] Lustre: lustre-MDT0000: new disk, initializing [ 5452.983103] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5452.985893] Lustre: Skipped 3 previous similar messages [ 5452.991729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5454.316798] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5458.647463] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 5458.647463] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5458.647463] [ 5458.657581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5458.707223] Lustre: 257311:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5458.736059] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5458.740177] Lustre: Skipped 1 previous similar message [ 5458.784750] Lustre: lustre-MDT0001: new disk, initializing [ 5458.848411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5458.852082] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5460.233428] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5464.334604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5464.449602] Lustre: lustre-OST0000: new disk, initializing [ 5464.451564] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5464.454388] Lustre: 258896:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5465.916430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5465.919698] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5465.953940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5466.343097] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5470.406111] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5470.466387] Lustre: lustre-OST0001: new disk, initializing [ 5470.469199] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5470.472485] Lustre: 259768:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5472.456209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5472.459929] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5472.497818] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5472.613732] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5476.957335] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:50:34 (1786485034) [ 5480.305845] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:50:37 (1786485037) [ 5491.681410] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5491.681567] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5491.688663] LustreError: Skipped 1 previous similar message [ 5491.692975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5491.698769] Lustre: Skipped 4 previous similar messages [ 5497.732281] Lustre: server umount lustre-OST0000 complete [ 5498.340607] LustreError: 258881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5498.347266] LustreError: 258881:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 5499.873042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5505.305403] Lustre: server umount lustre-MDT0000 complete [ 5506.492281] LustreError: 258107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485063 with bad export cookie 17777050397520087894 [ 5506.496761] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5506.500665] LustreError: 258107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5508.577332] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5508.581251] Lustre: Skipped 4 previous similar messages [ 5512.718146] Lustre: server umount lustre-MDT0001 complete [ 5516.306859] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5586.115420] Lustre: server umount lustre-OST0001 complete [ 5588.217452] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 5594.348630] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5605.936482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5606.021314] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5606.030988] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5606.064562] Lustre: lustre-MDT0000: new disk, initializing [ 5606.088641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5606.091699] Lustre: Skipped 3 previous similar messages [ 5606.096972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5607.359958] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5611.416956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5611.464684] Lustre: 264537:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5611.578528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5612.982214] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5616.669337] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5618.450298] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5620.649026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5620.753480] Lustre: 266507:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5622.522703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5622.526905] Lustre: Skipped 1 previous similar message [ 5622.529913] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5622.564379] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5622.585604] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5626.104204] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5632.993592] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5632.995103] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5632.996068] LustreError: Skipped 1 previous similar message [ 5632.996620] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5632.998930] Lustre: Skipped 1 previous similar message [ 5633.002351] Lustre: Skipped 9 previous similar messages [ 5634.513977] Lustre: server umount lustre-OST0000 complete [ 5641.817133] Lustre: server umount lustre-MDT0000 complete [ 5642.952265] LustreError: 266506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485200 with bad export cookie 17777050397520089364 [ 5642.954901] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5642.955763] LustreError: 266506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5643.075456] Lustre: server umount lustre-MDT0001 complete [ 5646.435229] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5647.435629] Key type lgssc unregistered [ 5647.560434] LNet: 268265:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5647.562815] LNetError: 268265:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5647.572364] LNet: Removed LNI 192.168.202.106@tcp [ 5647.893085] Key type .llcrypt unregistered [ 5647.893897] Key type ._llcrypt unregistered [ 5657.112626] Key type ._llcrypt registered [ 5657.113544] Key type .llcrypt registered [ 5657.154695] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5657.545866] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5657.586529] alg: No test for adler32 (adler32-zlib) [ 5658.441065] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 5658.522688] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5660.096136] Key type lgssc registered [ 5660.434813] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5662.875718] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5662.881884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5664.015981] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5665.379697] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5668.262690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5668.376598] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5669.721976] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5672.143344] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5674.011296] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5676.178833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5676.293571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5678.132936] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5680.545721] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5680.876413] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5683.682652] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5683.683984] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5683.688787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5687.567490] Lustre: server umount lustre-OST0000 complete [ 5688.775950] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5688.776324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5688.781610] Lustre: Skipped 1 previous similar message [ 5693.920974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5693.923150] Lustre: Skipped 2 previous similar messages [ 5699.041053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5699.042826] Lustre: Skipped 1 previous similar message [ 5703.136110] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5703.193324] Lustre: server umount lustre-MDT0000 complete [ 5704.161150] LustreError: 271137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5704.167048] LustreError: 271137:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5704.433848] LustreError: 269684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485261 with bad export cookie 13084054473786349383 [ 5704.436789] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5704.439423] LustreError: 269684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5704.552460] Lustre: server umount lustre-MDT0001 complete [ 5708.029064] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5708.993451] Key type lgssc unregistered [ 5709.120382] LNet: 273294:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5709.122408] LNetError: 273294:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5709.135282] LNet: Removed LNI 192.168.202.106@tcp [ 5709.428749] Key type .llcrypt unregistered [ 5709.430095] Key type ._llcrypt unregistered [ 5715.772285] Key type ._llcrypt registered [ 5715.773221] Key type .llcrypt registered [ 5715.806892] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5716.236578] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5716.247407] alg: No test for adler32 (adler32-zlib) [ 5717.086932] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 5717.165945] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5718.744160] Key type lgssc registered [ 5719.057754] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5721.833342] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5721.839622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5737.312448] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5742.432318] LustreError: 274465:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.106@tcp: failed processing log, type 4: rc = -110 [ 5769.056471] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5770.838559] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5773.110660] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5773.819770] Lustre: server umount lustre-OST0000 complete [ 5779.081600] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 5780.112543] Key type lgssc unregistered [ 5780.247449] LNet: 276096:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5780.250583] LNetError: 276096:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5780.263347] LNet: Removed LNI 192.168.202.106@tcp [ 5780.606332] Key type .llcrypt unregistered [ 5780.607323] Key type ._llcrypt unregistered [ 5794.146612] Key type ._llcrypt registered [ 5794.147398] Key type .llcrypt registered [ 5794.179577] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5794.470591] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5794.477152] alg: No test for adler32 (adler32-zlib) [ 5795.329897] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 5795.409946] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 5796.984150] Key type lgssc registered [ 5797.310982] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5801.573483] loop0: detected capacity change from 0 to 204800000 [ 5801.630033] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5801.633689] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5801.755876] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.088527] loop0: detected capacity change from 0 to 204800000 [ 5802.144533] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.147994] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.269853] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.602719] loop0: detected capacity change from 0 to 204800000 [ 5802.660597] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.663917] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5802.708017] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5803.056188] loop0: detected capacity change from 0 to 204800000 [ 5803.112762] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5803.493610] loop0: detected capacity change from 0 to 204800000 [ 5803.504828] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5804.830995] loop0: detected capacity change from 0 to 204800000 [ 5804.839229] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5806.121466] loop0: detected capacity change from 0 to 204800000 [ 5806.129238] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5807.432084] loop0: detected capacity change from 0 to 204800000 [ 5807.442305] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5808.730771] loop0: detected capacity change from 0 to 204800000 [ 5808.757918] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5808.768070] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5808.803897] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5808.808370] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5808.811471] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5808.814414] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5808.817567] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5808.820755] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5808.823959] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5808.827339] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5808.830469] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5808.833577] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5808.836432] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5808.839920] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5808.843732] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5808.848615] Lustre: 279489:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5811.405875] Lustre: server umount MGS complete [ 5811.707600] loop0: detected capacity change from 0 to 204800000 [ 5811.740811] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5811.897298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5811.913451] Lustre: lustre-MDT0000: Aborting client recovery [ 5812.234743] loop1: detected capacity change from 0 to 204800000 [ 5812.267412] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5812.312255] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5812.316905] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5812.321044] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5812.325491] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5812.330378] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5812.336943] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5812.342375] Lustre: 279919:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5812.350569] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5812.415349] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5812.439848] Lustre: lustre-MDT0001: Aborting client recovery [ 5812.761757] loop2: detected capacity change from 0 to 204800000 [ 5812.790384] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5812.819967] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5812.824602] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5812.828742] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5812.833727] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5812.838946] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5812.843345] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5812.848950] Lustre: 280001:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5812.902815] Lustre: Found index 0 for lustre-OST0000, updating log [ 5812.913413] Lustre: 280018:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5812.937297] Lustre: lustre-OST0000: Aborting recovery [ 5813.266496] loop3: detected capacity change from 0 to 204800000 [ 5813.298456] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5813.332689] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5813.336705] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5813.341420] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5813.345867] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5813.349661] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5813.352858] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5813.356676] Lustre: 280084:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5818.855624] LustreError: 279863:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5821.921980] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5821.922526] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5821.922914] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5821.927148] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5821.938992] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5821.942332] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5824.049870] LustreError: 279829:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 170us: rc = -2 [ 5837.280981] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5837.281255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5837.284638] Lustre: Skipped 3 previous similar messages [ 5837.286220] Lustre: Skipped 2 previous similar messages [ 5841.494108] Lustre: server umount lustre-MDT0000 complete [ 5841.801583] LustreError: 279812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485399 with bad export cookie 1445651553029632566 [ 5841.805498] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5842.400776] LustreError: 279828:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5842.400963] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5842.406115] LustreError: 279828:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5842.410539] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5842.415013] Lustre: Skipped 2 previous similar messages [ 5846.432519] LustreError: 279826:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5846.432545] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5846.437385] LustreError: 279826:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5846.440138] Lustre: Skipped 1 previous similar message [ 5851.552624] LustreError: 280327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5851.552669] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5851.557594] LustreError: 280327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5851.560232] Lustre: Skipped 1 previous similar message [ 5856.224130] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5856.313631] Lustre: server umount lustre-MDT0001 complete [ 5862.732777] Lustre: server umount lustre-OST0000 complete [ 5869.398535] Lustre: server umount lustre-OST0001 complete [ 5880.522798] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:57:16 (1786485436) [ 5905.611885] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 5910.347665] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 5929.778116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5929.926945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5929.946282] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5929.982303] Lustre: lustre-MDT0000: new disk, initializing [ 5930.022471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5930.026543] Lustre: Skipped 2 previous similar messages [ 5930.035500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5932.214685] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5937.831833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5937.880414] Lustre: 284153:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5937.914408] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5937.916379] Lustre: Skipped 1 previous similar message [ 5937.958590] Lustre: lustre-MDT0001: new disk, initializing [ 5937.990678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5938.010830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5938.015264] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5939.812076] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5944.222186] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5946.565384] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5949.623496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5949.745384] Lustre: lustre-OST0000: new disk, initializing [ 5949.747285] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5949.750715] Lustre: 286123:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5949.753578] Lustre: 286123:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 5949.792498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5952.466856] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 5956.992585] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5957.645777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5957.649776] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5957.697021] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5965.058150] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5974.630488] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5979.599511] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5985.377654] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6029.285457] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6029.286581] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6029.287188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6029.288829] LustreError: Skipped 1 previous similar message [ 6029.298536] Lustre: Skipped 2 previous similar messages [ 6039.008135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6039.083909] Lustre: server umount lustre-OST0000 complete [ 6044.641193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6044.642444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6044.646244] Lustre: Skipped 1 previous similar message [ 6044.648281] Lustre: Skipped 3 previous similar messages [ 6055.394112] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6055.498495] Lustre: server umount lustre-MDT0000 complete [ 6057.251518] LustreError: 284147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485614 with bad export cookie 1445651553029635268 [ 6057.252593] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6057.257393] LustreError: 284147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 6057.400364] Lustre: server umount lustre-MDT0001 complete [ 6061.231201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6062.998219] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6065.156562] Lustre: server umount MGS complete [ 6070.286969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6070.464184] LustreError: 290129:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6070.473671] LustreError: 290142:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6070.500544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6072.051938] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6075.506886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6077.203208] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6080.071550] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6082.306740] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6085.066743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6087.943033] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6090.990697] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6116.322097] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6116.325563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6116.331104] Lustre: Skipped 1 previous similar message [ 6116.331449] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6116.334207] Lustre: Skipped 6 previous similar messages [ 6122.049888] Lustre: server umount lustre-OST0000 complete [ 6126.561823] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6126.572398] Lustre: Skipped 1 previous similar message [ 6138.336117] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6138.401719] Lustre: server umount lustre-MDT0000 complete [ 6140.329492] LustreError: 290112:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485697 with bad export cookie 1445651553029637095 [ 6140.332807] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6140.337632] LustreError: 290112:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6140.488450] Lustre: server umount lustre-MDT0001 complete [ 6145.620807] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 6146.822818] Key type lgssc unregistered [ 6146.958707] LNet: 294088:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6146.962652] LNetError: 294088:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6146.970510] LNet: Removed LNI 192.168.202.106@tcp [ 6147.346849] Key type .llcrypt unregistered [ 6147.348066] Key type ._llcrypt unregistered [ 6162.516348] Key type ._llcrypt registered [ 6162.517751] Key type .llcrypt registered [ 6162.566857] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 6170.063859] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6170.501423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6170.557437] alg: No test for adler32 (adler32-zlib) [ 6171.438959] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 6171.551390] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 6173.152164] Key type lgssc registered [ 6173.656748] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6189.984106] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6190.000647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6191.107218] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6191.123875] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6191.167230] Lustre: lustre-MDT0000: new disk, initializing [ 6191.206703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6191.220950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6193.065922] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6198.121068] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6198.169452] Lustre: 298016:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6198.204376] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6198.206455] Lustre: Skipped 1 previous similar message [ 6198.260498] Lustre: lustre-MDT0001: new disk, initializing [ 6198.307388] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6198.322631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6198.326929] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6200.073815] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6204.783065] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6207.365814] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6210.337068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6210.450854] Lustre: lustre-OST0000: new disk, initializing [ 6210.453470] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6210.457158] Lustre: 299985:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6210.494899] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6212.604879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6212.609573] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6212.661159] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6213.032887] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6217.315910] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6221.242868] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6229.886521] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6237.986030] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6279.139103] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6279.140900] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6279.142858] LustreError: Skipped 1 previous similar message [ 6279.143642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6279.147530] Lustre: Skipped 1 previous similar message [ 6279.154372] Lustre: Skipped 1 previous similar message [ 6283.483589] Lustre: server umount lustre-OST0000 complete [ 6285.280803] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6285.283592] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6285.288866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6289.377145] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6289.377665] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6294.497151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6294.499958] Lustre: Skipped 2 previous similar messages [ 6298.593379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6298.596372] Lustre: Skipped 1 previous similar message [ 6299.104075] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6299.168762] Lustre: server umount lustre-MDT0000 complete [ 6300.569213] LustreError: 298008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485858 with bad export cookie 12549619533218407400 [ 6300.572769] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6300.574370] LustreError: 298008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6300.695469] Lustre: server umount lustre-MDT0001 complete [ 6303.998255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6305.517924] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6307.258164] Lustre: server umount MGS complete [ 6311.393481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6311.560792] LustreError: 304054:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6311.585824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6313.026397] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6316.197054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6316.340550] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6317.844144] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6320.577211] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6322.666331] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6325.141069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6325.264122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6327.390656] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6330.128690] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6357.985480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6357.987555] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6357.991235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6363.671190] Lustre: server umount lustre-OST0000 complete [ 6367.712905] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6367.717324] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6367.723775] Lustre: Skipped 1 previous similar message [ 6371.096413] Lustre: server umount lustre-MDT0000 complete [ 6372.617195] LustreError: 304778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786485930 with bad export cookie 12549619533218409234 [ 6372.620442] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6372.621799] LustreError: 304778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6372.750923] Lustre: server umount lustre-MDT0001 complete [ 6376.831699] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 6377.915264] Key type lgssc unregistered [ 6378.047519] LNet: 308008:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6378.050707] LNetError: 308008:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6378.059361] LNet: Removed LNI 192.168.202.106@tcp [ 6378.381204] Key type .llcrypt unregistered [ 6378.382565] Key type ._llcrypt unregistered [ 6386.124472] Key type ._llcrypt registered [ 6386.125518] Key type .llcrypt registered [ 6386.175561] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 6392.650085] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6393.025091] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6393.081544] alg: No test for adler32 (adler32-zlib) [ 6393.991635] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 6394.106802] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 6395.704150] Key type lgssc registered [ 6396.175906] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6404.935794] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6410.414997] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6410.424987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6411.511779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6411.524837] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6411.562677] Lustre: lustre-MDT0000: new disk, initializing [ 6411.585140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6411.592620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6412.999059] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6417.497520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6417.602596] Lustre: lustre-OST0000: new disk, initializing [ 6417.604924] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6417.607436] Lustre: Skipped 1 previous similar message [ 6417.610828] Lustre: 312031:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6417.646941] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6419.801506] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6421.375581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6421.378437] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6421.387867] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6656.993440] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6656.998393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6660.463906] Lustre: server umount lustre-MDT0000 complete [ 6661.780247] LustreError: 311217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486219 with bad export cookie 11340700755654647859 [ 6661.782301] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6661.784119] LustreError: 311217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6661.848857] Lustre: server umount lustre-OST0000 complete [ 6670.606688] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 6674.104528] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6687.201112] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6687.283492] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6687.294368] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6687.329502] Lustre: lustre-MDT0000: new disk, initializing [ 6687.353232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6687.360548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6688.781911] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6692.952509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6692.999445] Lustre: 316268:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6693.025454] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6693.028153] Lustre: Skipped 1 previous similar message [ 6693.067070] Lustre: lustre-MDT0001: new disk, initializing [ 6693.102315] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6693.114502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6693.118601] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6694.494596] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6698.526121] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6700.709658] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6703.259784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6703.369382] Lustre: lustre-OST0000: new disk, initializing [ 6703.371243] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6703.374585] Lustre: 318236:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6703.408540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6705.550072] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6709.456206] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6710.778759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6710.783078] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6710.812608] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6711.030819] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6711.111044] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6712.590540] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6712.669907] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6715.873663] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6715.874236] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6715.876882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6715.877774] LustreError: Skipped 1 previous similar message [ 6715.884518] Lustre: Skipped 1 previous similar message [ 6719.504511] Lustre: server umount lustre-OST0000 complete [ 6720.993359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6720.994060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6720.998105] Lustre: Skipped 1 previous similar message [ 6720.999778] Lustre: Skipped 1 previous similar message [ 6726.115280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6726.118996] Lustre: Skipped 1 previous similar message [ 6731.232936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6731.234556] Lustre: Skipped 1 previous similar message [ 6735.328140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6735.395219] Lustre: server umount lustre-MDT0000 complete [ 6736.779405] LustreError: 317062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486294 with bad export cookie 11340700755657588636 [ 6736.782851] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6736.784358] LustreError: 317062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6736.904749] Lustre: server umount lustre-MDT0001 complete [ 6738.107891] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6738.710138] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 18:11:36 (1786486296) [ 6741.081743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6741.264493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6742.739240] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6745.962751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6747.536656] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6750.061760] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6751.876198] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6754.098800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6756.072543] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6758.753801] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6761.215831] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6761.280857] Lustre: lustre-OST0001: new disk, initializing [ 6761.283833] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6761.288438] Lustre: 323379:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6761.318568] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6761.320840] Lustre: Skipped 2 previous similar messages [ 6762.363667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6762.366460] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6762.381969] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6763.324569] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6763.427477] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6763.430618] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 6764.451459] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6764.456315] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6766.503941] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6766.508933] LustreError: 323554:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 6767.206829] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6767.280451] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6769.074361] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff995ec6fce000.ost_server_uuid 50 [ 6769.582325] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff995ec6fce000.ost_server_uuid in FULL state after 0 sec [ 6770.596881] LustreError: 323559:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6770.600572] LustreError: 323559:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 6771.425062] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff995ec6fce000.ost_server_uuid 50 [ 6771.931442] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff995ec6fce000.ost_server_uuid in FULL state after 0 sec [ 6772.648772] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6786.122314] Lustre: lustre-OST0001: Not available for connect from 192.168.202.6@tcp (stopping) [ 6787.046683] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6789.694547] Lustre: server umount lustre-OST0001 complete [ 6792.163557] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6792.164206] LustreError: 322364:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6792.164563] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6792.164569] Lustre: Skipped 1 previous similar message [ 6792.176987] LustreError: 322364:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 6797.285157] LustreError: 322366:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6797.292525] LustreError: 322366:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6797.337541] Lustre: server umount lustre-OST0000 complete [ 6802.402840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6802.403979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6802.405278] LustreError: Skipped 1 previous similar message [ 6802.409295] Lustre: Skipped 1 previous similar message [ 6802.414138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6802.416454] Lustre: Skipped 7 previous similar messages [ 6804.759446] Lustre: server umount lustre-MDT0000 complete [ 6806.069146] LustreError: 320530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486363 with bad export cookie 11340700755657589504 [ 6806.072020] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6806.073109] LustreError: 320530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6806.197989] Lustre: server umount lustre-MDT0001 complete [ 6809.849317] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 6810.932758] Key type lgssc unregistered [ 6811.066389] LNet: 326056:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6811.069681] LNetError: 326056:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6811.077330] LNet: Removed LNI 192.168.202.106@tcp [ 6811.388063] Key type .llcrypt unregistered [ 6811.389304] Key type ._llcrypt unregistered [ 6820.616229] Key type ._llcrypt registered [ 6820.617369] Key type .llcrypt registered [ 6820.659526] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6820.976706] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6820.981927] alg: No test for adler32 (adler32-zlib) [ 6821.836463] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 6821.923086] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 6823.504159] Key type lgssc registered [ 6823.881698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6826.661769] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6826.667691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6827.814946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6829.142525] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6831.745132] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6834.155678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6834.282890] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6835.635788] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6838.901611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6839.060530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6841.239837] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6843.832382] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6844.196814] LustreError: 329816:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6844.203363] LustreError: 329816:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6846.295082] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6846.377421] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6848.423203] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6850.980632] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6852.586339] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6852.586709] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6857.820538] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6857.906713] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6933.472865] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6933.476022] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6933.479597] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6934.499389] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6934.499543] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6934.500343] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6934.501084] Lustre: Skipped 1 previous similar message [ 6939.512775] Lustre: server umount lustre-OST0001 complete [ 6939.620083] LustreError: 329130:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6939.624309] LustreError: 329130:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6941.666044] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6941.668396] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6941.673296] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6944.738782] LustreError: 329130:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6944.738867] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6944.739677] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6944.741156] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6944.743012] LustreError: 329130:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6944.751233] Lustre: Skipped 1 previous similar message [ 6947.149331] Lustre: server umount lustre-OST0000 complete [ 6949.858247] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6949.859439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6949.863753] Lustre: Skipped 1 previous similar message [ 6954.591306] Lustre: server umount lustre-MDT0000 complete [ 6954.977474] LustreError: 329044:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6954.984077] LustreError: 329044:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6955.976362] LustreError: 327433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486513 with bad export cookie 6175848805784736598 [ 6955.978371] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6955.981176] LustreError: 327433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6956.106874] Lustre: server umount lustre-MDT0001 complete [ 6959.793572] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 6960.880770] Key type lgssc unregistered [ 6961.003657] LNet: 332389:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6961.007759] LNetError: 332389:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6961.017376] LNet: Removed LNI 192.168.202.106@tcp [ 6961.355935] Key type .llcrypt unregistered [ 6961.357030] Key type ._llcrypt unregistered [ 6979.847438] Key type ._llcrypt registered [ 6979.848837] Key type .llcrypt registered [ 6979.888808] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 6980.225940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6980.288247] alg: No test for adler32 (adler32-zlib) [ 6981.124853] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 6981.214370] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 6982.792157] Key type lgssc registered [ 6983.128913] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6986.260733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6986.267091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6987.426079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6988.903669] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6991.891898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6992.031564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6993.406768] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 6994.379527] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6996.582078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6996.708405] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6997.731832] LustreError: 336141:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6997.735664] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6997.738306] LustreError: 336141:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6998.702484] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7001.550031] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7001.626072] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7002.670122] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7003.696544] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7007.888074] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7015.733261] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7023.076726] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7023.076909] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7023.077782] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7023.086177] Lustre: Skipped 3 previous similar messages [ 7025.497617] Lustre: server umount lustre-MDT0000 complete [ 7026.740342] LustreError: 334637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486584 with bad export cookie 4992441394372430516 [ 7026.744076] LustreError: 334637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7026.744168] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7026.855628] Lustre: server umount lustre-MDT0001 complete [ 7038.406341] Lustre: server umount lustre-OST0000 complete [ 7049.956914] Lustre: server umount lustre-OST0001 complete [ 7055.583289] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7059.272515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7059.409792] LustreError: 339912:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7059.427606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7060.864337] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7063.499525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7065.026690] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7066.023487] Lustre: 341054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7068.259398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7068.401171] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7068.404166] Lustre: Skipped 1 previous similar message [ 7069.412784] LustreError: 341405:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7069.415574] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7070.411742] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7073.419048] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7074.542986] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7075.502669] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7078.528741] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7079.770553] Lustre: 342921:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7084.000829] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7084.003454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7084.011628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7084.013242] Lustre: Skipped 3 previous similar messages [ 7085.026961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7085.028498] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7085.031826] Lustre: Skipped 2 previous similar messages [ 7086.871456] Lustre: server umount lustre-MDT0000 complete [ 7087.994263] LustreError: 339893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486645 with bad export cookie 4992441394372432175 [ 7087.996806] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7087.997699] LustreError: 339893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7088.104120] Lustre: server umount lustre-MDT0001 complete [ 7099.334353] Lustre: server umount lustre-OST0000 complete [ 7110.811954] Lustre: server umount lustre-OST0001 complete [ 7111.894549] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7112.430375] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7112.976029] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:17:50 (1786486670) [ 7120.011580] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7122.745424] loop0: detected capacity change from 0 to 1099511627776 [ 7122.929517] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7122.932824] *** VALIDATE xfs *** [ 7122.936920] XFS (loop0): Mounting V5 Filesystem [ 7122.945388] XFS (loop0): Ending clean mount [ 7123.358360] loop1: detected capacity change from 0 to 18691697672192 [ 7142.467920] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 7145.318602] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7156.628271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7156.720653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7156.731703] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7156.769994] Lustre: lustre-MDT0000: new disk, initializing [ 7156.793580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7156.795863] Lustre: Skipped 1 previous similar message [ 7156.801841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7158.170663] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7162.474873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7162.521438] Lustre: 348030:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7162.544132] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7162.545679] Lustre: Skipped 1 previous similar message [ 7162.580256] Lustre: lustre-MDT0001: new disk, initializing [ 7162.626212] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7162.629603] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7164.015659] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7167.724587] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7169.651843] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7172.033117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7172.139997] Lustre: lustre-OST0000: new disk, initializing [ 7172.141762] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7172.144701] Lustre: 349997:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7173.433745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7173.436980] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7173.478364] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7174.151482] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7177.891823] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7179.337307] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7179.408491] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7180.799588] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7180.873730] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7183.843554] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7183.844044] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7183.844464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7183.844469] Lustre: Skipped 3 previous similar messages [ 7183.846729] Lustre: Skipped 1 previous similar message [ 7183.852116] LustreError: Skipped 1 previous similar message [ 7187.671559] Lustre: server umount lustre-OST0000 complete [ 7193.569059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7193.572406] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7193.576876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7193.579061] Lustre: Skipped 1 previous similar message [ 7203.296106] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7203.359925] Lustre: server umount lustre-MDT0000 complete [ 7204.321456] LustreError: 348042:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7204.329754] LustreError: 348042:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7204.640115] LustreError: 348824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486762 with bad export cookie 4992441394372433708 [ 7204.643320] LustreError: 348824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7204.643862] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7204.771373] Lustre: server umount lustre-MDT0001 complete [ 7207.597173] XFS (loop0): Unmounting Filesystem [ 7209.271822] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:19:26 (1786486766) [ 7211.524761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7211.674086] LustreError: 352552:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7211.696207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7211.699131] Lustre: Skipped 2 previous similar messages [ 7213.014288] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7215.977243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7217.348987] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7219.517977] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7221.198970] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7223.276760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7225.205201] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7227.555323] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7242.210618] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7242.210912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7242.215732] Lustre: Skipped 1 previous similar message [ 7242.218344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7242.221704] Lustre: Skipped 5 previous similar messages [ 7243.283772] Lustre: server umount lustre-OST0000 complete [ 7246.615511] Lustre: server umount lustre-MDT0000 complete [ 7246.817369] LustreError: 352571:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7247.843445] LustreError: 352534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786486805 with bad export cookie 4992441394372434590 [ 7247.846738] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7247.848495] LustreError: 352534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7247.954220] Lustre: server umount lustre-MDT0001 complete [ 7251.417062] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 7252.415806] Key type lgssc unregistered [ 7252.549354] LNet: 356202:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7252.551392] LNetError: 356202:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7252.558250] LNet: Removed LNI 192.168.202.106@tcp [ 7252.854119] Key type .llcrypt unregistered [ 7252.855529] Key type ._llcrypt unregistered [ 7262.402254] Key type ._llcrypt registered [ 7262.403167] Key type .llcrypt registered [ 7262.441854] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 7262.722071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7262.727113] alg: No test for adler32 (adler32-zlib) [ 7263.576105] Lustre: Lustre: Build Version: 2.17.56_51_g4226bd4 [ 7263.659276] LNet: Added LNI 192.168.202.106@tcp [8/256/0/180] [ 7265.232163] Key type lgssc registered [ 7265.556032] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7268.062144] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7268.067984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7269.198926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7270.415965] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7272.977107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7273.088239] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7274.275704] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7276.495537] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7278.271818] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7280.453347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7280.570576] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7282.358938] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7284.488403] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7299.040911] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7299.043440] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7299.048055] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7300.067434] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7300.067647] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7303.774553] Lustre: server umount lustre-MDT0001 complete [ 7305.186192] LustreError: 357744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7305.193187] LustreError: 357744:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7307.460070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7307.495158] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7307.497085] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7307.510328] Lustre: 357733:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7307.639468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7309.105607] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7312.869828] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7313.895264] LustreError: 361476:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7313.899620] LustreError: 361476:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7313.903215] LustreError: 357736:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7313.922873] LustreError: 361479:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -17 [ 7313.925674] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7313.925674] [ 7612.488968] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.6@tcp (stopping) [ 7612.491970] Lustre: Skipped 2 previous similar messages [ 7614.945378] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7614.946808] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7614.948995] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7614.953953] Lustre: Skipped 1 previous similar message [ 7617.366079] Lustre: server umount lustre-MDT0001 complete [ 7617.602424] LustreError: 359241:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7617.607809] LustreError: 359241:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7620.066033] LustreError: 358483:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7620.070992] LustreError: 358483:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7620.779499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7620.809523] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7620.811095] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7620.821970] Lustre: 357733:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7620.925613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7622.189621] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7626.214316] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7630.311656] LustreError: 374199:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7630.314699] LustreError: 374199:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7630.317328] LustreError: 357736:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7630.319730] LustreError: 357736:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7630.337750] LustreError: 374202:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 7630.339921] LustreError: 374202:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 7630.341623] LustreError: 374202:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -17 [ 7630.344083] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.106@tcp [ 7630.344083] [ 7924.809272] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.6@tcp (stopping) [ 7924.812830] Lustre: Skipped 1 previous similar message [ 7928.293361] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7928.293741] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7930.526283] Lustre: server umount lustre-MDT0001 complete [ 7933.409181] LustreError: 357743:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7933.413988] LustreError: 357743:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7933.993947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7934.027613] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7934.030759] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7934.047927] Lustre: 357732:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7934.171480] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7935.551934] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 7939.558714] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7943.142382] LustreError: 386828:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7943.145143] LustreError: 386828:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7943.147563] LustreError: 357736:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7943.150068] LustreError: 357736:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7943.168968] LustreError: 386831:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7943.174488] LustreError: 386831:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.106@tcp: cfg command failed: rc = -114 [ 7943.177492] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7943.177492] [ 8251.875386] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8251.875947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8251.876642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8251.876645] Lustre: Skipped 3 previous similar messages [ 8251.879906] Lustre: Skipped 2 previous similar messages [ 8264.160097] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8264.223950] Lustre: server umount lustre-MDT0000 complete [ 8265.454511] LustreError: 357725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786487822 with bad export cookie 4884958099591311193 [ 8265.460322] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8267.233151] LustreError: 362232:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8267.233375] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8267.244452] Lustre: Skipped 1 previous similar message [ 8270.241332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8270.243330] Lustre: Skipped 9 previous similar messages [ 8271.657278] Lustre: server umount lustre-MDT0001 complete [ 8273.010605] Lustre: server umount lustre-OST0000 complete [ 8276.003095] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 8279.195663] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8291.188188] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:37:28 (1786487848) [ 8293.397879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8293.482409] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8293.491566] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8293.532053] Lustre: lustre-MDT0000: new disk, initializing [ 8293.552790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8293.558678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8294.873342] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8299.057843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8299.104282] Lustre: 401383:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8299.135182] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8299.137659] Lustre: Skipped 1 previous similar message [ 8299.179406] Lustre: lustre-MDT0001: new disk, initializing [ 8299.217140] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8299.229870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8299.234527] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8300.584072] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8304.321133] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8306.095435] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8308.299344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8308.406671] Lustre: lustre-OST0000: new disk, initializing [ 8308.408459] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8308.411428] Lustre: 403351:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8308.440387] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8308.719558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8308.722391] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8308.758881] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8310.419956] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_default_debug -1 all [ 8314.086861] Lustre: DEBUG MARKER: oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8318.947121] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8318.948173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8318.951162] Lustre: Skipped 2 previous similar messages [ 8318.953189] Lustre: Skipped 1 previous similar message [ 8322.921255] Lustre: server umount lustre-MDT0000 complete [ 8324.065197] LustreError: 402193:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8324.070714] LustreError: 402193:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8324.146424] LustreError: 401377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786487881 with bad export cookie 4884958099591313132 [ 8324.149302] LustreError: 402190:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8324.149675] LustreError: MGC192.168.202.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8324.150900] LustreError: 401377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8324.269368] Lustre: server umount lustre-MDT0001 complete [ 8335.778991] Lustre: server umount lustre-OST0000 complete [ 8345.280361] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing set_hostid [ 8348.279368] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing load_modules_local [ 8357.594437] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8216 sec ========== 18:38:35 (1786487915) [ 8358.095291] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:38:35 (1786487915) === [ 8359.128310] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:38:36 (1786487916) === [ 8371.090418] Lustre: DEBUG MARKER: oleg206-server.virtnet: executing unload_modules_local [ 8372.027384] Key type lgssc unregistered [ 8372.148315] LNet: 409708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8372.150177] LNetError: 409708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8372.161246] LNet: Removed LNI 192.168.202.106@tcp [ 8372.464210] Key type .llcrypt unregistered [ 8372.465251] Key type ._llcrypt unregistered