[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 466787815 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2496MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003375] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008480] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011132] LSM: Security Framework initializing [ 0.013006] Yama: becoming mindful. [ 0.014039] SELinux: Initializing. [ 0.015065] *** VALIDATE selinux *** [ 0.023739] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028097] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029160] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030108] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031110] *** VALIDATE tmpfs *** [ 0.032460] *** VALIDATE proc *** [ 0.033248] *** VALIDATE cgroup *** [ 0.034009] *** VALIDATE cgroup2 *** [ 0.036163] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037158] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039031] Spectre V2 : User space: Vulnerable [ 0.040011] Speculative Store Bypass: Vulnerable [ 0.042844] debug: unmapping init [mem 0xffffffffaec59000-0xffffffffaec60fff] [ 0.044171] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045671] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046024] ... version: 2 [ 0.047027] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049014] ... value mask: 0000ffffffffffff [ 0.050018] ... max period: 00007fffffffffff [ 0.051015] ... fixed-purpose events: 3 [ 0.052016] ... event mask: 000000070000000f [ 0.053310] rcu: Hierarchical SRCU implementation. [ 0.055432] smp: Bringing up secondary CPUs ... [ 0.056582] x86: Booting SMP configuration: [ 0.057025] .... node #0, CPUs: #1 #2 #3 [ 0.060359] smp: Brought up 1 node, 4 CPUs [ 0.062014] smpboot: Max logical packages: 1 [ 0.063018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.129000] node 0 deferred pages initialised in 65ms [ 0.135106] devtmpfs: initialized [ 0.136258] x86/mm: Memory block size: 128MB [ 0.139870] gcov: version magic: 0x41383552 [ 0.142286] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.143071] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.144253] pinctrl core: initialized pinctrl subsystem [ 0.145220] [ 0.145970] ************************************************************* [ 0.146014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.147010] ** ** [ 0.148012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.149012] ** ** [ 0.150016] ** This means that this kernel is built to expose internal ** [ 0.151012] ** IOMMU data structures, which may compromise security on ** [ 0.152014] ** your system. ** [ 0.153014] ** ** [ 0.154012] ** If you see this message and you are not debugging the ** [ 0.155013] ** kernel, report this immediately to your vendor! ** [ 0.156014] ** ** [ 0.157016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.158012] ************************************************************* [ 0.159723] NET: Registered protocol family 16 [ 0.160444] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.161058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.162053] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.163465] cpuidle: using governor menu [ 0.166032] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.169505] PCI: Using configuration type 1 for base access [ 0.172137] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.182066] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.185033] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.189056] cryptd: max_cpu_qlen set to 1000 [ 0.194250] ACPI: Added _OSI(Module Device) [ 0.196014] ACPI: Added _OSI(Processor Device) [ 0.197011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.199013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.203937] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.210433] ACPI: Interpreter enabled [ 0.212054] ACPI: PM: (supports S0 S3 S4 S5) [ 0.213010] ACPI: Using IOAPIC for interrupt routing [ 0.215089] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.219393] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.231617] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.234035] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.238035] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.244071] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.251404] acpiphp: Slot [2] registered [ 0.253113] acpiphp: Slot [5] registered [ 0.254066] acpiphp: Slot [6] registered [ 0.256133] acpiphp: Slot [7] registered [ 0.257091] acpiphp: Slot [8] registered [ 0.259090] acpiphp: Slot [9] registered [ 0.260110] acpiphp: Slot [10] registered [ 0.262114] acpiphp: Slot [3] registered [ 0.263101] acpiphp: Slot [4] registered [ 0.265074] acpiphp: Slot [11] registered [ 0.266064] acpiphp: Slot [12] registered [ 0.267066] acpiphp: Slot [13] registered [ 0.268091] acpiphp: Slot [14] registered [ 0.269055] acpiphp: Slot [15] registered [ 0.271088] acpiphp: Slot [16] registered [ 0.273128] acpiphp: Slot [17] registered [ 0.274099] acpiphp: Slot [18] registered [ 0.276098] acpiphp: Slot [19] registered [ 0.278076] acpiphp: Slot [20] registered [ 0.280126] acpiphp: Slot [21] registered [ 0.282125] acpiphp: Slot [22] registered [ 0.284107] acpiphp: Slot [23] registered [ 0.286104] acpiphp: Slot [24] registered [ 0.288110] acpiphp: Slot [25] registered [ 0.290102] acpiphp: Slot [26] registered [ 0.292095] acpiphp: Slot [27] registered [ 0.294118] acpiphp: Slot [28] registered [ 0.296104] acpiphp: Slot [29] registered [ 0.298093] acpiphp: Slot [30] registered [ 0.299100] acpiphp: Slot [31] registered [ 0.301064] PCI host bridge to bus 0000:00 [ 0.302018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.304020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.306030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.309024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.311017] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.314022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.316203] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.317849] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.321363] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.332880] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.336785] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.339021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.342017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.346019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.349147] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.351666] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.354034] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.357732] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.362012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.374016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.381016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.389905] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.402017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.411016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.441015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.453671] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.467014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.474000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.490016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.501882] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.512015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.525015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.542015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.553360] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.561018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.568014] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.592015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.606464] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.615022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.626023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.651016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.661452] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.668016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.675015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.689025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.698721] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.701446] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.704440] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.707362] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.709260] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.714038] iommu: Default domain type: Passthrough [ 0.715473] SCSI subsystem initialized [ 0.717149] ACPI: bus type USB registered [ 0.719130] usbcore: registered new interface driver usbfs [ 0.721079] usbcore: registered new interface driver hub [ 0.723072] usbcore: registered new device driver usb [ 0.724182] pps_core: LinuxPPS API ver. 1 registered [ 0.726018] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.730082] PTP clock support registered [ 0.732062] EDAC MC: Ver: 3.0.0 [ 0.734147] PCI: Using ACPI for IRQ routing [ 0.736616] NetLabel: Initializing [ 0.738024] NetLabel: domain hash size = 128 [ 0.740021] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.742106] NetLabel: unlabeled traffic allowed by default [ 0.745185] vgaarb: loaded [ 0.747229] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.749014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.756148] clocksource: Switched to clocksource kvm-clock [ 0.871548] VFS: Disk quotas dquot_6.6.0 [ 0.873364] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.876167] *** VALIDATE ramfs *** [ 0.877635] *** VALIDATE hugetlbfs *** [ 0.879361] pnp: PnP ACPI init [ 0.881798] pnp: PnP ACPI: found 6 devices [ 0.899652] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.903537] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.906157] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.908816] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.911414] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.914333] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.917326] NET: Registered protocol family 2 [ 0.920182] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.925709] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.929857] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.935416] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.939267] TCP: Hash tables configured (established 65536 bind 65536) [ 0.942646] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.946165] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.949376] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.952765] NET: Registered protocol family 1 [ 0.955634] RPC: Registered named UNIX socket transport module. [ 0.958075] RPC: Registered udp transport module. [ 0.959530] RPC: Registered tcp transport module. [ 0.961333] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.964665] NET: Registered protocol family 44 [ 0.966448] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.968802] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.971112] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.973564] PCI: CLS 0 bytes, default 64 [ 0.975307] Unpacking initramfs... [ 2.384264] debug: unmapping init [mem 0xffff9e4cbcc54000-0xffff9e4cbffbffff] [ 2.389262] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.391461] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.394431] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.907673] Initialise system trusted keyrings [ 2.909840] Key type blacklist registered [ 2.911768] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.922381] zbud: loaded [ 2.926420] *** VALIDATE nfs *** [ 2.928230] *** VALIDATE nfs4 *** [ 2.929864] pstore: using deflate compression [ 2.932989] Platform Keyring initialized [ 3.032629] NET: Registered protocol family 38 [ 3.034474] Key type asymmetric registered [ 3.036198] Asymmetric key parser 'x509' registered [ 3.038593] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.042177] io scheduler mq-deadline registered [ 3.043643] io scheduler kyber registered [ 3.045063] io scheduler bfq registered [ 3.046441] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.049239] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.052076] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.055127] ACPI: Power Button [PWRF] [ 3.060606] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.068177] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.086283] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.093498] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.111585] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.140817] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.171486] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.177981] Non-volatile memory driver v1.3 [ 3.179959] Linux agpgart interface v0.103 [ 3.212489] virtio_blk virtio1: [vda] 149424 512-byte logical blocks (76.5 MB/73.0 MiB) [ 3.216838] vda: detected capacity change from 0 to 76505088 [ 3.237543] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.240132] vdb: detected capacity change from 0 to 1073741824 [ 3.255216] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.257958] vdc: detected capacity change from 0 to 2621440000 [ 3.274895] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.277893] vdd: detected capacity change from 0 to 2621440000 [ 3.293354] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.296189] vde: detected capacity change from 0 to 4294967296 [ 3.310824] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.314037] vdf: detected capacity change from 0 to 4294967296 [ 3.324142] libphy: Fixed MDIO Bus: probed [ 3.329212] usbcore: registered new interface driver usbserial_generic [ 3.332292] usbserial: USB Serial support registered for generic [ 3.335086] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.339133] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.340953] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.343539] mousedev: PS/2 mouse device common for all mice [ 3.347317] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.351450] rtc_cmos 00:05: RTC can wake from S4 [ 3.355633] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.356705] rtc_cmos 00:05: registered as rtc0 [ 3.365528] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.367052] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.373704] intel_pstate: CPU model not supported [ 3.376902] hid: raw HID events driver (C) Jiri Kosina [ 3.394345] usbcore: registered new interface driver usbhid [ 3.397024] usbhid: USB HID core driver [ 3.402706] drop_monitor: Initializing network drop monitor service [ 3.409554] Initializing XFRM netlink socket [ 3.412051] NET: Registered protocol family 10 [ 3.415518] Segment Routing with IPv6 [ 3.417255] NET: Registered protocol family 17 [ 3.419771] mpls_gso: MPLS GSO support [ 3.429306] RAS: Correctable Errors collector initialized. [ 3.431941] AVX version of gcm_enc/dec engaged. [ 3.434505] AES CTR mode by8 optimization enabled [ 3.521617] sched_clock: Marking stable (3521510881, 0)->(4478152343, -956641462) [ 3.525862] registered taskstats version 1 [ 3.528333] Loading compiled-in X.509 certificates [ 3.530711] zswap: loaded using pool lzo/zbud [ 3.557521] Key type big_key registered [ 3.570596] Key type encrypted registered [ 3.572354] ima: No TPM chip found, activating TPM-bypass! [ 3.574656] ima: Allocated hash algorithm: sha1 [ 3.576599] ima: No architecture policies found [ 3.578630] evm: Initialising EVM extended attributes: [ 3.580787] evm: security.selinux [ 3.582167] evm: security.ima [ 3.583227] evm: security.capability [ 3.584716] evm: HMAC attrs: 0x1 [ 3.588283] rtc_cmos 00:05: setting system clock to 2026-08-22 08:33:03 UTC (1787387583) [ 3.595348] debug: unmapping init [mem 0xffffffffafc03000-0xffffffffafdfffff] [ 3.598663] debug: unmapping init [mem 0xffffffffae982000-0xffffffffaec58fff] [ 3.609103] Write protecting the kernel read-only data: 28672k [ 3.612652] debug: unmapping init [mem 0xffffffffad003000-0xffffffffad1fffff] [ 3.616197] debug: unmapping init [mem 0xffffffffad914000-0xffffffffad9fffff] [ 3.664200] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.672803] systemd[1]: Detected virtualization kvm. [ 3.675281] systemd[1]: Detected architecture x86-64. [ 3.677304] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.704211] systemd[1]: No hostname configured. [ 3.706246] systemd[1]: Set hostname to . [ 3.708373] random: systemd: uninitialized urandom read (16 bytes read) [ 3.710519] systemd[1]: Initializing machine ID from random generator. [ 3.760461] random: ln: uninitialized urandom read (6 bytes read) [ 3.858762] random: systemd: uninitialized urandom read (16 bytes read) [ 3.862204] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.867188] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.873726] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.544332] device-mapper: uevent: version 1.0.3 [ 4.547137] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.313063] random: fast init done [ 5.350054] virtio_net virtio0 ens2: renamed from eth0 [ 5.441601] scsi host0: ata_piix [ 5.572294] scsi host1: ata_piix [ 5.575071] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.577254] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.133989] dracut-initqueue[585]: RTNETLINK answers: File exists [ 10.092244] random: crng init done [ 10.094036] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.581869] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.821541] printk: systemd: 24 output lines suppressed due to ratelimiting [ 12.109693] SELinux: Disabled at runtime. [ 12.172067] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.179580] systemd[1]: Detected virtualization kvm. [ 12.180942] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.737795] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.741299] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.748439] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.752257] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.755988] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.766984] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.773937] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd File Systems. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 12.986771] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.299359] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.628889] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.630790] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.830914] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.842701] EDAC sbridge: Ver: 1.1.2 [ 15.429101] Key type dns_resolver registered [ 15.745995] NFS: Registering the id_resolver key type [ 15.748361] Key type id_resolver registered [ 15.750239] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg624-server login: [ 44.318823] libcfs: loading out-of-tree module taints kernel. [ 44.356840] Key type ._llcrypt registered [ 44.359189] Key type .llcrypt registered [ 44.448433] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 61.685319] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 63.178591] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 63.193340] alg: No test for adler32 (adler32-zlib) [ 64.614873] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 65.554150] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 67.399325] Key type lgssc registered [ 67.868123] hrtimer: interrupt took 9114504 ns [ 69.008143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 87.519126] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 129.172046] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 142.426301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 142.476691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 143.904854] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 143.963493] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 144.125760] Lustre: lustre-MDT0000: new disk, initializing [ 144.244147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 144.263677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 148.588878] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 162.263255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 162.376479] Lustre: 6507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 162.401876] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 162.407828] Lustre: Skipped 1 previous similar message [ 162.470516] Lustre: lustre-MDT0001: new disk, initializing [ 162.540554] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 162.567094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 162.585879] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 167.762131] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 174.005203] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 188.080761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 188.385241] Lustre: lustre-OST0000: new disk, initializing [ 188.390747] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 188.408397] Lustre: 8446:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 188.467742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.487299] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.662691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.676849] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.730567] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 213.897142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 214.089541] Lustre: lustre-OST0001: new disk, initializing [ 214.097202] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 214.111622] Lustre: 9519:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 214.190304] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 220.676525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 220.689667] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 220.782273] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 221.193105] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.745243] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 244.127476] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 250.468584] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing check_logdir /tmp/testlogs/ [ 256.781198] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing yml_node [ 260.815599] Lustre: DEBUG MARKER: Client: 2.17.54.111 [ 263.024962] Lustre: DEBUG MARKER: MDS: 2.17.54.111 [ 265.180196] Lustre: DEBUG MARKER: OSS: 2.17.54.111 [ 267.171825] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 04:37:24 EDT 2026 [ 282.616757] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 283.975483] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 301.023703] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 301.039713] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 301.064770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 302.563938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 302.565076] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 305.524822] Lustre: server umount lustre-MDT0000 complete [ 309.353389] LustreError: 6499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787387889 with bad export cookie 5628356341857343224 [ 309.359913] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 309.363773] LustreError: 6499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 312.800154] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 312.801318] LustreError: 7817:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 312.807651] Lustre: Skipped 2 previous similar messages [ 312.808165] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 312.808169] Lustre: Skipped 3 previous similar messages [ 312.840709] LustreError: 7817:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 314.847606] LustreError: 6512:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 314.848043] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 314.875054] Lustre: Skipped 2 previous similar messages [ 315.964220] Lustre: server umount lustre-MDT0001 complete [ 319.963375] Lustre: server umount lustre-OST0000 complete [ 323.486515] Lustre: server umount lustre-OST0001 complete [ 329.221833] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 338.980896] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 373.189734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 373.391774] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 373.413926] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 373.521162] Lustre: lustre-MDT0000: new disk, initializing [ 373.606928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 373.622883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 377.740438] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 387.735240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 387.847851] Lustre: 15069:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 387.919970] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 387.933130] Lustre: Skipped 1 previous similar message [ 388.061328] Lustre: lustre-MDT0001: new disk, initializing [ 388.181603] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 388.213548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 388.227667] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 392.353356] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 400.913870] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 405.675106] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 411.631573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 411.995752] Lustre: lustre-OST0000: new disk, initializing [ 412.002115] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 412.008786] Lustre: 17038:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 412.123433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 413.175715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 413.184541] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 413.281153] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 418.042486] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 426.331130] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 432.713114] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 432.883863] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 438.126861] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 438.488676] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 443.873516] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 443.873739] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 443.890300] LustreError: Skipped 1 previous similar message [ 443.892245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 443.932708] Lustre: Skipped 2 previous similar messages [ 446.863340] Lustre: server umount lustre-OST0000 complete [ 454.112533] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 454.125173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 454.129074] Lustre: Skipped 2 previous similar messages [ 456.882653] Lustre: server umount lustre-MDT0000 complete [ 459.233281] LustreError: 15075:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 459.250704] LustreError: 15075:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 460.517933] LustreError: 15061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388040 with bad export cookie 5628356341857344757 [ 460.518648] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 460.523510] LustreError: 15061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 460.853176] Lustre: server umount lustre-MDT0001 complete [ 466.522692] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:40:44 (1787388044) [ 473.163953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 473.498886] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 473.587421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 478.693255] LustreError: 19303:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.656118] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 488.931162] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 488.957920] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 492.368527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 498.957833] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 507.018706] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 514.300820] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 521.824199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 528.924847] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 536.815656] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 552.107165] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 559.584334] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 559.591944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 559.602178] Lustre: Skipped 1 previous similar message [ 559.609612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 565.823976] Lustre: server umount lustre-MDT0000 complete [ 569.322421] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.356428] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 569.603810] LustreError: 19284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388149 with bad export cookie 5628356341857345639 [ 569.617532] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 569.976763] Lustre: server umount lustre-MDT0001 complete [ 583.998112] Lustre: server umount lustre-OST0000 complete [ 606.139819] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 618.210406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 618.685619] LustreError: 23703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 618.770781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 618.784346] Lustre: Skipped 2 previous similar messages [ 623.716711] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 633.252101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.885339] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 642.384815] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 649.819874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 656.885947] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 666.216687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 666.399885] Lustre: lustre-OST0001: new disk, initializing [ 666.403712] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 666.410106] Lustre: 26021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 667.838476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 667.852589] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 667.953850] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 673.030853] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 683.385946] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 686.957659] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 700.392484] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 700.409186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 700.421207] Lustre: Skipped 2 previous similar messages [ 700.429957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 700.432583] Lustre: Skipped 3 previous similar messages [ 706.336694] Lustre: server umount lustre-MDT0000 complete [ 709.092087] LustreError: 23703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 709.115693] LustreError: 23703:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 710.196533] LustreError: 23683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388290 with bad export cookie 5628356341857346479 [ 710.198881] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 710.204368] LustreError: 23683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 710.623911] Lustre: server umount lustre-MDT0001 complete [ 724.794870] Lustre: server umount lustre-OST0000 complete [ 739.098393] Lustre: server umount lustre-OST0001 complete [ 757.173517] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 767.443427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 768.003679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 768.012873] Lustre: Skipped 3 previous similar messages [ 772.650847] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 781.249922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 787.349965] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 791.157518] Lustre: 30190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 798.260829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 804.261030] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 812.854460] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 813.030738] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 813.035612] Lustre: Skipped 5 previous similar messages [ 819.796644] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 828.894646] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 833.350314] Lustre: 32067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 843.231900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 843.243303] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 843.256729] Lustre: Skipped 3 previous similar messages [ 846.085428] Lustre: server umount lustre-MDT0000 complete [ 849.383130] LustreError: 31834:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 849.399565] LustreError: 31834:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 849.760525] LustreError: 29025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388429 with bad export cookie 5628356341857347781 [ 849.785904] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 849.989505] Lustre: server umount lustre-MDT0001 complete [ 863.939774] Lustre: server umount lustre-OST0000 complete [ 878.210954] Lustre: server umount lustre-OST0001 complete [ 886.989262] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:47:44 (1787388464) [ 914.892887] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 924.995448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 929.810903] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 939.621358] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 945.658158] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 949.397403] Lustre: 36018:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 956.773917] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 963.839915] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 973.063733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 980.723384] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 988.952382] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 992.734539] Lustre: 37899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 999.396612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 999.405238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 999.408166] Lustre: Skipped 5 previous similar messages [ 999.413788] Lustre: Skipped 6 previous similar messages [ 1004.649392] Lustre: server umount lustre-MDT0000 complete [ 1008.558454] LustreError: 34851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388588 with bad export cookie 5628356341857348887 [ 1008.562699] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1008.567560] LustreError: 34851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1009.044652] Lustre: server umount lustre-MDT0001 complete [ 1023.513200] Lustre: server umount lustre-OST0000 complete [ 1026.015594] Lustre: 3639:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787388589/real 1787388589] req@ffff9e4c051bed80 x1874211787594368/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787388605 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1027.401879] Lustre: server umount lustre-OST0001 complete [ 1036.104633] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:50:13 (1787388613) [ 1050.758715] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1059.709815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1060.179514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1060.184231] Lustre: Skipped 7 previous similar messages [ 1064.066661] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1071.647613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1075.613234] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1078.579436] Lustre: 41190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1085.200422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1091.609096] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1099.611326] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1106.931769] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1116.182864] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1120.087582] Lustre: 43065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1120.803248] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1120.814393] Lustre: Skipped 1 previous similar message [ 1123.298991] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1129.676169] Lustre: server umount lustre-MDT0000 complete [ 1129.849205] LustreError: 40038:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1129.883391] LustreError: 40038:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1133.644340] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388713 with bad export cookie 5628356341857349986 [ 1133.655970] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1133.675209] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1136.099338] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1136.132757] Lustre: Skipped 7 previous similar messages [ 1140.313830] Lustre: server umount lustre-MDT0001 complete [ 1147.784344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1158.635121] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x4e1bf045c9b975d0 [ 1158.651634] Lustre: MGC192.168.206.124@tcp: Connection restored to 0@lo (at 0@lo) [ 1162.581524] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1168.242413] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1170.152389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1174.480336] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1175.546666] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1175.559953] Lustre: Skipped 1 previous similar message [ 1182.511031] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1184.925708] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1191.783641] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1193.725361] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1201.125805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1205.674962] Lustre: server umount lustre-MDT0000 complete [ 1209.035038] LustreError: 40022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388788 with bad export cookie 5628356341857351120 [ 1209.039775] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1209.053974] LustreError: 40022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1209.398827] Lustre: server umount lustre-MDT0001 complete [ 1224.001891] Lustre: server umount lustre-OST0000 complete [ 1238.071589] Lustre: server umount lustre-OST0001 complete [ 1245.902585] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:53:43 (1787388823) [ 1259.943447] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1268.876593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1273.460287] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1280.532203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1285.311711] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1288.005808] Lustre: 48831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1294.418816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1300.943418] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1310.097487] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1316.257601] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1324.539927] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1328.824542] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1340.404176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1340.411045] Lustre: Skipped 22 previous similar messages [ 1342.221819] Lustre: server umount lustre-MDT0000 complete [ 1345.360381] LustreError: 47659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388925 with bad export cookie 5628356341857351911 [ 1345.371443] LustreError: 47659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1345.376774] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1345.650959] Lustre: server umount lustre-MDT0001 complete [ 1354.862444] Lustre: server umount lustre-OST0000 complete [ 1358.030751] Lustre: server umount lustre-OST0001 complete [ 1366.427195] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:55:43 (1787388943) [ 1368.819223] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1370.455916] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:55:48 (1787388948) [ 1379.479829] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 1387.161639] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1404.022782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1404.224470] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1404.295896] Lustre: lustre-MDT0000: new disk, initializing [ 1404.392878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1407.773897] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1415.537729] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1421.055607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1421.243541] Lustre: lustre-OST0000: new disk, initializing [ 1421.247276] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1421.253218] Lustre: Skipped 1 previous similar message [ 1421.258694] Lustre: 54899:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1422.486000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1422.498680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1422.531176] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1427.840576] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1437.620316] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1439.890944] Lustre: DEBUG MARKER: create test files [ 1474.016901] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1474.018195] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1474.027054] Lustre: Skipped 10 previous similar messages [ 1478.093172] Lustre: server umount lustre-OST0000 complete [ 1482.102819] Lustre: server umount lustre-MDT0000 complete [ 1492.148091] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 1494.858470] Key type lgssc unregistered [ 1495.185756] LNet: 56531:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1495.203679] LNetError: 56531:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1496.231479] LNet: Removed LNI 192.168.206.124@tcp [ 1497.067154] Key type .llcrypt unregistered [ 1497.070367] Key type ._llcrypt unregistered [ 1521.139568] Key type ._llcrypt registered [ 1521.145453] Key type .llcrypt registered [ 1521.286693] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1522.259041] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1522.324667] alg: No test for adler32 (adler32-zlib) [ 1523.424743] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 1523.672703] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 1525.359162] Key type lgssc registered [ 1526.467501] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1533.928688] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1533.952095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1535.455042] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1539.775523] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1547.399748] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1553.487427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1553.890131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1559.269966] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1561.065951] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1561.125869] Lustre: 59242:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1561.135801] Lustre: 59242:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1561.144287] Lustre: 59242:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1561.150241] Lustre: 59242:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1561.157254] Lustre: 59242:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1561.161820] Lustre: 59242:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1565.049758] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1570.041319] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1575.508568] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1584.374870] Lustre: server umount lustre-OST0000 complete [ 1598.042050] Lustre: server umount lustre-MDT0000 complete [ 1607.134753] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 1609.487348] Key type lgssc unregistered [ 1609.731914] LNet: 60850:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1609.745567] LNetError: 60850:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1609.765813] LNet: Removed LNI 192.168.206.124@tcp [ 1610.396481] Key type .llcrypt unregistered [ 1610.401231] Key type ._llcrypt unregistered [ 1640.884105] Key type ._llcrypt registered [ 1640.892289] Key type .llcrypt registered [ 1641.038543] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1642.470939] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1642.508189] alg: No test for adler32 (adler32-zlib) [ 1643.575852] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 1643.865890] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 1645.591175] Key type lgssc registered [ 1646.660114] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1654.932059] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1654.955283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1656.319969] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1660.063870] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1667.702761] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1674.697996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1674.964233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1680.467234] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1681.153225] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1681.223642] Lustre: 63693:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1681.238388] Lustre: 63693:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1681.244460] Lustre: 63693:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1681.265616] Lustre: 63693:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1681.273558] Lustre: 63693:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1681.282034] Lustre: 63693:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1687.430712] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1692.310327] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1701.858604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1701.861483] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1701.870570] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1706.359400] Lustre: server umount lustre-OST0000 complete [ 1709.924379] Lustre: server umount lustre-MDT0000 complete [ 1718.562770] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 1720.643647] Key type lgssc unregistered [ 1720.907025] LNet: 65261:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1720.914864] LNetError: 65261:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1720.929621] LNet: Removed LNI 192.168.206.124@tcp [ 1721.636947] Key type .llcrypt unregistered [ 1721.643904] Key type ._llcrypt unregistered [ 1737.063414] Key type ._llcrypt registered [ 1737.069638] Key type .llcrypt registered [ 1737.189745] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 1750.935678] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1751.644690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1751.655320] alg: No test for adler32 (adler32-zlib) [ 1752.692349] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 1752.896475] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 1754.559168] Key type lgssc registered [ 1755.613047] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1784.649597] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1784.677712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1785.843570] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1785.880868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1785.988479] Lustre: lustre-MDT0000: new disk, initializing [ 1786.050721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1786.069401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1789.903539] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1799.511268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1799.609601] Lustre: 68402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1799.654737] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1799.658941] Lustre: Skipped 1 previous similar message [ 1799.749987] Lustre: lustre-MDT0001: new disk, initializing [ 1799.872584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1799.932141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1799.947934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1804.559252] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1813.718079] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1819.510343] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1826.506234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1826.801314] Lustre: lustre-OST0000: new disk, initializing [ 1826.806278] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1826.810987] Lustre: 70371:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1826.883204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1832.518201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1832.533132] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1832.675950] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1833.168530] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1841.933350] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1846.406425] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1846.700850] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1851.742656] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1851.867352] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1858.532966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1858.533729] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1858.539475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1858.552558] Lustre: Skipped 1 previous similar message [ 1858.558908] LustreError: Skipped 1 previous similar message [ 1859.876525] Lustre: server umount lustre-OST0000 complete [ 1863.653070] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1863.658892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1863.675265] Lustre: Skipped 1 previous similar message [ 1863.680393] Lustre: Skipped 1 previous similar message [ 1868.770821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1868.776051] Lustre: Skipped 2 previous similar messages [ 1869.209355] Lustre: server umount lustre-MDT0000 complete [ 1872.684910] LustreError: 68397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389452 with bad export cookie 14768725485558409712 [ 1872.702057] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1872.703918] LustreError: 68397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1873.076911] Lustre: server umount lustre-MDT0001 complete [ 1880.758292] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:04:18 (1787389458) [ 1891.351413] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1931.535948] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 1941.645515] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 1974.954198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1975.253312] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1975.302234] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1975.403466] Lustre: lustre-MDT0000: new disk, initializing [ 1975.531708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1975.565256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1979.428541] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 1989.088585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1989.198341] Lustre: 76392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1989.266338] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1989.275935] Lustre: Skipped 1 previous similar message [ 1989.283025] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1989.363719] Lustre: lustre-MDT0001: new disk, initializing [ 1989.456576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1989.504722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1989.516649] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1993.386977] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2002.517399] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2007.890471] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2014.264834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2014.556320] Lustre: lustre-OST0000: new disk, initializing [ 2014.560482] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2014.573250] Lustre: 78359:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2014.709450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2021.066720] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2021.429036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2021.438149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2021.583053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2030.052419] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2034.650816] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2034.930702] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2039.266987] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2039.454697] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2041.828645] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2041.836272] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2041.849000] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2047.597979] Lustre: server umount lustre-OST0000 complete [ 2052.581421] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2052.597679] Lustre: Skipped 1 previous similar message [ 2052.603652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2052.608650] Lustre: Skipped 3 previous similar messages [ 2057.552185] Lustre: server umount lustre-MDT0000 complete [ 2057.705280] LustreError: 77525:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2057.735083] LustreError: 77525:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2061.498200] LustreError: 79698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389641 with bad export cookie 14768725485558410594 [ 2061.507122] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2061.519104] LustreError: 79698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2061.936870] Lustre: server umount lustre-MDT0001 complete [ 2070.511648] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:07:27 (1787389647) [ 2076.613571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2077.069867] LustreError: 80771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2077.120559] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2081.079175] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2082.273232] LustreError: 80772:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2086.369077] LustreError: 80771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2087.809242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2092.579948] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2098.825663] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2104.244094] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2110.339532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2110.595266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2110.602488] Lustre: Skipped 1 previous similar message [ 2115.652328] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2123.190509] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2126.618366] Lustre: 83441:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting ab63561b-fc9b-46ce-8bf0-e5f46762ef58 at adminstrative request [ 2127.329479] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2127.353586] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2137.582343] Lustre: MGS: Client ab63561b-fc9b-46ce-8bf0-e5f46762ef58 (at 0@lo) reconnecting [ 2137.604509] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xccf50d94bff7b0d4 to 0xccf50d94bff7b374 [ 2137.623126] Lustre: MGC192.168.206.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2163.065936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2163.238851] Lustre: lustre-OST0001: new disk, initializing [ 2163.242852] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2163.247235] Lustre: 83834:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2168.514568] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2170.436400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2170.445447] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2170.541987] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2176.694595] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2181.096759] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2181.101024] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2181.110632] Lustre: Skipped 1 previous similar message [ 2181.111617] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2181.125116] LustreError: Skipped 2 previous similar messages [ 2193.375109] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2193.461133] Lustre: server umount lustre-OST0001 complete [ 2196.453964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2196.460836] LustreError: 82588:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2196.468447] Lustre: Skipped 2 previous similar messages [ 2196.491697] LustreError: 82588:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2202.515668] Lustre: server umount lustre-OST0000 complete [ 2206.176260] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2211.906808] Lustre: server umount lustre-MDT0000 complete [ 2215.026075] LustreError: 80753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389794 with bad export cookie 14768725485558412148 [ 2215.031624] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2215.036249] LustreError: 80753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2215.272760] Lustre: server umount lustre-MDT0001 complete [ 2222.592372] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:10:00 (1787389800) [ 2224.129946] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2225.478704] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:10:03 (1787389803) [ 2227.050663] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2228.690870] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:10:06 (1787389806) [ 2230.150483] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2232.084616] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:10:09 (1787389809) [ 2233.701777] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2235.283184] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:10:12 (1787389812) [ 2236.960351] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2242.324543] loop: module loaded [ 2242.350071] loop0: detected capacity change from 0 to 4294967296 [ 2242.397189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2244.132654] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2247.313363] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2250.766789] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2276.007877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2276.031489] LustreError: 87956:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2276.040609] LustreError: 87956:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2276.048183] LustreError: 87956:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2276.053143] LustreError: 87956:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2293.564630] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 2302.137697] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 2333.842764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2334.054935] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2334.081768] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2334.151235] Lustre: lustre-MDT0000: new disk, initializing [ 2334.230500] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2334.235048] Lustre: Skipped 1 previous similar message [ 2334.248796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2338.182680] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2347.446046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2347.581737] Lustre: 90822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2347.851212] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2352.070571] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2361.843789] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2367.439375] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2374.538155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2374.764890] Lustre: 92792:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2380.365425] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2380.486432] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2380.820459] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2389.806141] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2394.466677] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2394.684063] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2399.218246] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2399.431309] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2406.373485] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2406.378480] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2406.385205] Lustre: Skipped 2 previous similar messages [ 2406.389683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2406.412302] Lustre: Skipped 14 previous similar messages [ 2407.570178] Lustre: server umount lustre-OST0000 complete [ 2425.311779] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2425.455846] Lustre: server umount lustre-MDT0000 complete [ 2426.848806] LustreError: 90832:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2426.861044] LustreError: 90832:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2428.728643] LustreError: 92791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390008 with bad export cookie 14768725485558412764 [ 2428.734629] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2428.745956] LustreError: 92791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2429.089606] Lustre: server umount lustre-MDT0001 complete [ 2437.407257] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:13:34 (1787390014) [ 2443.397140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2443.866173] LustreError: 95204:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2448.419586] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2456.086577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2460.550660] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2467.841259] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2472.859919] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2480.447498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2480.822717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2480.829243] Lustre: Skipped 4 previous similar messages [ 2487.004510] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2494.211702] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2499.185670] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2499.407677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2503.609764] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2503.867812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2510.677212] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2510.906424] Lustre: lustre-OST0001: new disk, initializing [ 2510.908704] Lustre: Skipped 2 previous similar messages [ 2510.911636] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2510.914730] Lustre: Skipped 3 previous similar messages [ 2510.918702] Lustre: 98574:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2517.762365] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2518.107637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2518.114665] Lustre: Skipped 2 previous similar messages [ 2518.118243] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2518.222990] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2528.055538] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2533.019666] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2533.292957] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2538.219672] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2538.434108] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2558.960764] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2584.302065] Lustre: Failing over lustre-MDT0000 [ 2584.551209] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2584.560478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2584.575261] Lustre: Skipped 3 previous similar messages [ 2584.580722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2584.587573] Lustre: Skipped 6 previous similar messages [ 2586.726424] Lustre: server umount lustre-MDT0000 complete [ 2587.023752] LustreError: 95199:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2587.039729] LustreError: 95199:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2606.559122] Lustre: 66628:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787390170/real 1787390170] req@00000000854dd5ed x1874213556457984/t0(0) o400->MGC192.168.206.124@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787390186 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2606.608413] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2611.697411] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2611.700437] LDISKFS-fs (dm-0): recovery complete [ 2611.709094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2616.821702] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xccf50d94bff87def [ 2616.831617] Lustre: MGC192.168.206.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2617.291714] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2617.720820] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2617.726968] Lustre: 95201:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2622.061719] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2622.448327] Lustre: 95204:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2622.463361] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2622.466275] Lustre: 95204:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2622.550878] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2622.891708] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2623.229889] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2623.240346] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2623.559207] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2623.570261] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2623.583047] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2623.595908] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2624.248069] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2624.258700] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2624.583401] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2624.594926] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2625.892651] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2625.912624] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 2626.573140] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2626.585040] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2626.903104] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2626.912713] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2630.090982] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2630.109498] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2630.745580] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2630.757620] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2631.083078] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2631.087894] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2638.139081] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2638.149102] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 2638.790485] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2638.807088] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2639.127611] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2639.137791] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2654.394130] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2654.409884] Lustre: 101423:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2655.056963] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2655.066419] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2655.396213] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2655.407878] LustreError: 101423:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2677.314057] Lustre: 101423:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2677.328561] Lustre: 101423:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2677.345211] Lustre: 101423:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client dcb148ef-e8ff-4947-a401-13c75d81b42b@192.168.206.24@tcp [ 2677.354937] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2677.361489] Lustre: 101423:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@0000000062a6a60a x1874213541579776/t0(8589934768) o35->dcb148ef-e8ff-4947-a401-13c75d81b42b@192.168.206.24@tcp:247/0 lens 392/0 e 0 to 0 dl 1787390267 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2677.397632] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2677.415938] Lustre: lustre-MDT0000: Denying connection for new client dcb148ef-e8ff-4947-a401-13c75d81b42b (at 192.168.206.24@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2677.421425] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2677.489807] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2677.497552] Lustre: Skipped 2 previous similar messages [ 2677.520972] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2677.582120] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2677.582511] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2697.185396] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2697.202727] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2697.218585] Lustre: Skipped 3 previous similar messages [ 2701.517828] Lustre: server umount lustre-OST0000 complete [ 2704.364549] LustreError: 97024:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2704.398162] LustreError: 97024:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2711.831931] Lustre: server umount lustre-OST0001 complete [ 2718.688902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2718.696754] Lustre: Skipped 12 previous similar messages [ 2722.496135] Lustre: server umount lustre-MDT0000 complete [ 2726.398273] LustreError: 97045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390306 with bad export cookie 14768725485558463983 [ 2726.408163] LustreError: 97045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2726.409237] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2726.958059] Lustre: server umount lustre-MDT0001 complete [ 2736.000526] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:18:33 (1787390313) [ 2744.585610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2744.590598] Lustre: *** cfs_fail_loc=197, val=0*** [ 2760.223698] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2765.344188] LustreError: 103398:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.124@tcp: failed processing log, type 4: rc = -110 [ 2770.482753] Lustre: *** cfs_fail_loc=197, val=0*** [ 2770.487708] Lustre: Skipped 65 previous similar messages [ 2791.007289] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2791.022629] Lustre: Skipped 2 previous similar messages [ 2798.144610] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2806.385995] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2808.659865] Lustre: server umount lustre-OST0000 complete [ 2817.756524] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:19:55 (1787390395) [ 2838.520937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2838.966389] LustreError: 105026:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2838.991548] LustreError: 105026:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2843.721283] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2853.705849] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2858.296668] LustreError: 105057:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2858.310582] LustreError: 105057:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2858.319569] LustreError: 105057:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 2858.635805] Lustre: server umount lustre-MDT0000 complete [ 2897.865126] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 2908.213352] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 2947.190939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2947.412465] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2947.473883] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2947.574617] Lustre: lustre-MDT0000: new disk, initializing [ 2947.692403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2952.005454] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2962.359670] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2962.490108] Lustre: 109231:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2962.780542] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2967.792554] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2977.260990] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2982.956131] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2989.488546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2989.719258] Lustre: 111200:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2995.945940] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 2998.087738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3005.769322] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3010.570669] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3010.850684] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3015.025206] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3015.191325] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3018.721024] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3018.725904] LustreError: Skipped 4 previous similar messages [ 3018.728178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3018.729504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3018.747141] Lustre: Skipped 6 previous similar messages [ 3018.754032] Lustre: Skipped 2 previous similar messages [ 3023.155453] Lustre: server umount lustre-OST0000 complete [ 3041.247466] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3041.508673] Lustre: server umount lustre-MDT0000 complete [ 3044.828494] LustreError: 109268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390624 with bad export cookie 14768725485558474553 [ 3044.837760] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3044.840915] LustreError: 109268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3045.247277] Lustre: server umount lustre-MDT0001 complete [ 3054.458395] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:23:51 (1787390631) [ 3064.766621] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 3067.386866] Key type lgssc unregistered [ 3067.719649] LNet: 113690:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3067.729851] LNetError: 113690:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3067.750368] LNet: Removed LNI 192.168.206.124@tcp [ 3068.523210] Key type .llcrypt unregistered [ 3068.527113] Key type ._llcrypt unregistered [ 3086.905291] Key type ._llcrypt registered [ 3086.907057] Key type .llcrypt registered [ 3087.080133] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 3105.025264] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 3106.058370] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3106.218651] alg: No test for adler32 (adler32-zlib) [ 3107.398642] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 3107.589068] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 3109.263256] Key type lgssc registered [ 3110.341734] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3151.605554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3151.658660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3152.886815] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3152.924407] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3152.982123] Lustre: lustre-MDT0000: new disk, initializing [ 3153.058363] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3153.080855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3156.884715] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3166.681322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3166.919987] Lustre: lustre-OST0000: new disk, initializing [ 3166.923145] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3166.926762] Lustre: Skipped 1 previous similar message [ 3166.930656] Lustre: 117863:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3166.991667] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3168.973095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3168.984951] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3169.011162] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3172.380134] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3182.412172] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3182.586525] Lustre: lustre-OST0001: new disk, initializing [ 3182.589249] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3182.594412] Lustre: 118725:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3182.682162] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3188.583116] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3192.961469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3192.970964] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3193.007822] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3203.041271] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3203.043368] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3203.059172] Lustre: Skipped 1 previous similar message [ 3203.073859] Lustre: Skipped 1 previous similar message [ 3204.448882] Lustre: server umount lustre-MDT0000 complete [ 3210.479230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3214.589554] LustreError: 117862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390794 with bad export cookie 8464645055190854195 [ 3214.590793] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3214.595012] LustreError: 117862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3214.659956] Lustre: server umount lustre-OST0000 complete [ 3218.419698] Lustre: server umount lustre-OST0001 complete [ 3226.350736] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:26:43 (1787390803) [ 3245.229665] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:27:02 (1787390822) [ 3253.795918] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 3274.306056] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:27:31 (1787390851) [ 3307.495841] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 3317.113239] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 3349.717256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3349.883767] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3349.909308] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3350.008626] Lustre: lustre-MDT0000: new disk, initializing [ 3350.089100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3350.112373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3354.369421] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3365.784994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3365.923754] Lustre: 126478:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3366.012429] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3366.018161] Lustre: Skipped 1 previous similar message [ 3366.131671] Lustre: lustre-MDT0001: new disk, initializing [ 3366.247666] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3366.290436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3366.304176] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3370.337402] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3379.717157] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3385.930117] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3393.134237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3393.380744] Lustre: lustre-OST0000: new disk, initializing [ 3393.384071] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3393.388101] Lustre: 128446:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3393.450565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3399.533033] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3399.731332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3399.740930] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3399.906747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3408.177697] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3412.314128] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3412.483528] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3417.105671] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3417.410172] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3420.646902] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3420.651695] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3420.660725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3425.767371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3425.781939] Lustre: Skipped 2 previous similar messages [ 3426.466418] Lustre: server umount lustre-OST0000 complete [ 3430.881260] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3430.904868] Lustre: Skipped 1 previous similar message [ 3430.912989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3430.920406] Lustre: Skipped 1 previous similar message [ 3436.000936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3436.734348] Lustre: server umount lustre-MDT0000 complete [ 3440.047741] LustreError: 126469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391019 with bad export cookie 8464645055190855420 [ 3440.052255] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3440.063255] LustreError: 126469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3440.409724] Lustre: server umount lustre-MDT0001 complete [ 3449.646812] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:30:26 (1787391026) [ 3456.471506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3457.258230] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3462.637492] LustreError: 130855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3462.663129] LustreError: 130855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3462.863261] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3467.748661] LustreError: 130856:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3472.874264] LustreError: 130855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3473.709846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3478.839480] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3487.554836] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3493.350661] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3500.032640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3500.462061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3500.476593] Lustre: Skipped 1 previous similar message [ 3507.205751] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3514.801524] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3523.044233] Lustre: *** cfs_fail_loc=159, val=0*** [ 3523.054016] Lustre: Skipped 1 previous similar message [ 3538.374791] Lustre: lustre-MDT0000: Client 300ea4a4-3707-434f-8bf2-ef6834b967c4 (at 192.168.206.24@tcp) reconnecting [ 3540.752310] Lustre: *** cfs_fail_loc=159, val=0*** [ 3540.761122] Lustre: Skipped 4 previous similar messages [ 3556.263524] Lustre: lustre-MDT0000: Client 300ea4a4-3707-434f-8bf2-ef6834b967c4 (at 192.168.206.24@tcp) reconnecting [ 3562.797956] Lustre: server umount lustre-OST0000 complete [ 3570.145157] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3570.149976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3570.162743] Lustre: Skipped 2 previous similar messages [ 3570.176404] Lustre: Skipped 2 previous similar messages [ 3576.478814] Lustre: server umount lustre-MDT0000 complete [ 3580.072050] LustreError: 130835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391159 with bad export cookie 8464645055190856302 [ 3580.073993] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3580.082447] LustreError: 130835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3589.418721] Lustre: server umount lustre-MDT0001 complete [ 3599.472301] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 3601.784278] Key type lgssc unregistered [ 3602.023639] LNet: 134630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3602.032651] LNetError: 134630:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3602.044431] LNet: Removed LNI 192.168.206.124@tcp [ 3602.817271] Key type .llcrypt unregistered [ 3602.819392] Key type ._llcrypt unregistered [ 3622.841958] Key type ._llcrypt registered [ 3622.845156] Key type .llcrypt registered [ 3622.931740] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 3623.957117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3624.002088] alg: No test for adler32 (adler32-zlib) [ 3625.006406] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 3625.211725] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 3626.887274] Key type lgssc registered [ 3627.977778] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3635.423356] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3635.435704] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3636.845747] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3640.723188] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3646.435032] LustreError: 136025:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3646.450271] LustreError: 136025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3649.046700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3649.466615] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3653.566430] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3661.311394] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3666.995314] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3673.551943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3674.002736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3680.516735] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3681.202560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3688.478555] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3699.044871] Lustre: *** cfs_fail_loc=159, val=0*** [ 3714.487990] Lustre: lustre-MDT0000: Client 301ff61f-ff26-4570-9238-9c8e70c21c0e (at 192.168.206.24@tcp) reconnecting [ 3717.111371] Lustre: *** cfs_fail_loc=159, val=0*** [ 3732.416995] Lustre: lustre-MDT0001: Client 301ff61f-ff26-4570-9238-9c8e70c21c0e (at 192.168.206.24@tcp) reconnecting [ 3734.177684] Lustre: *** cfs_fail_loc=159, val=0*** [ 3734.192304] Lustre: Skipped 4 previous similar messages [ 3750.865684] Lustre: lustre-MDT0001: Client 301ff61f-ff26-4570-9238-9c8e70c21c0e (at 192.168.206.24@tcp) reconnecting [ 3766.471506] Lustre: 139082:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3769.366198] Lustre: *** cfs_fail_loc=159, val=0*** [ 3769.380286] Lustre: Skipped 3 previous similar messages [ 3785.653582] Lustre: lustre-MDT0000: Client 2de952ef-c15d-43dd-8001-7a5b549e3d55 (at 192.168.206.24@tcp) reconnecting [ 3787.483415] Lustre: *** cfs_fail_loc=159, val=0*** [ 3787.490641] Lustre: Skipped 14 previous similar messages [ 3804.087147] Lustre: lustre-MDT0000: Client 2de952ef-c15d-43dd-8001-7a5b549e3d55 (at 192.168.206.24@tcp) reconnecting [ 3805.864479] Lustre: 139372:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3813.346693] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3813.368165] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3813.396479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3814.372779] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3814.374001] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3814.415212] Lustre: Skipped 1 previous similar message [ 3818.693585] Lustre: server umount lustre-OST0000 complete [ 3823.586832] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3823.598817] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3823.637352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3825.957952] Lustre: server umount lustre-MDT0000 complete [ 3829.740269] LustreError: 136021:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3829.774164] LustreError: 136021:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3830.331104] LustreError: 136005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391410 with bad export cookie 11931093601210946892 [ 3830.340385] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3830.348127] LustreError: 136005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3830.762809] Lustre: server umount lustre-MDT0001 complete [ 3843.987751] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 3846.084219] Key type lgssc unregistered [ 3846.332181] LNet: 140428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3846.336566] LNetError: 140428:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3846.352425] LNet: Removed LNI 192.168.206.124@tcp [ 3847.027461] Key type .llcrypt unregistered [ 3847.029556] Key type ._llcrypt unregistered [ 3874.668138] Key type ._llcrypt registered [ 3874.680332] Key type .llcrypt registered [ 3874.856600] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 3876.146141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3876.175694] alg: No test for adler32 (adler32-zlib) [ 3877.280608] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 3877.618078] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 3879.271172] Key type lgssc registered [ 3880.369053] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3888.130455] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3888.180171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3889.738416] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3894.241030] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3899.884385] LustreError: 141845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3899.897344] LustreError: 141845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3902.888729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3903.395184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3907.547648] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3915.453422] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3921.941714] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3929.931356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3930.298612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3934.441831] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3937.006727] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 3946.054670] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3948.806297] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3955.410841] Lustre: 144428:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3959.364566] Lustre: 144477:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3960.860625] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3961.780344] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3966.686454] Lustre: 144625:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3967.668715] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3968.755878] Lustre: 144724:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3974.624856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3974.628699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3974.653725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3974.657469] Lustre: Skipped 1 previous similar message [ 3979.475071] Lustre: server umount lustre-OST0000 complete [ 3984.872374] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3984.875785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3984.884290] Lustre: Skipped 2 previous similar messages [ 3984.896106] Lustre: Skipped 1 previous similar message [ 3989.984850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3989.997418] Lustre: Skipped 1 previous similar message [ 3995.108047] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3995.119073] Lustre: Skipped 1 previous similar message [ 3998.175377] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3998.349802] Lustre: server umount lustre-MDT0000 complete [ 4000.230836] LustreError: 141841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4000.251545] LustreError: 141841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4003.006672] LustreError: 141823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391582 with bad export cookie 8220779613260324056 [ 4003.007257] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4003.027536] LustreError: 141823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4003.428517] Lustre: server umount lustre-MDT0001 complete [ 4014.314983] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 4017.216859] Key type lgssc unregistered [ 4017.570821] LNet: 145785:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4017.585360] LNetError: 145785:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4017.602731] LNet: Removed LNI 192.168.206.124@tcp [ 4018.598594] Key type .llcrypt unregistered [ 4018.600545] Key type ._llcrypt unregistered [ 4044.140840] Key type ._llcrypt registered [ 4044.143243] Key type .llcrypt registered [ 4044.290696] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4045.572792] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4045.595399] alg: No test for adler32 (adler32-zlib) [ 4046.616481] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 4046.804255] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 4048.447546] Key type lgssc registered [ 4049.386479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4056.167949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4056.199350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4057.876776] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4063.393711] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4067.307733] LustreError: 147202:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4067.360312] LustreError: 147202:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4072.428170] LustreError: 147203:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4073.869463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4074.765088] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4079.327506] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4087.558133] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4094.768404] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4101.979228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4102.546619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4103.600794] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4109.831865] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4118.175901] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4119.984370] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4125.343147] Lustre: *** cfs_fail_loc=159, val=0*** [ 4142.012885] Lustre: lustre-MDT0000: Client b3309a53-6a92-4d32-9a0d-004f342221f7 (at 192.168.206.24@tcp) reconnecting [ 4148.197289] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4148.200265] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.206241] LustreError: Skipped 2 previous similar messages [ 4148.218728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4150.774241] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4150.778480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4150.784992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4150.806501] Lustre: Skipped 1 previous similar message [ 4153.423462] Lustre: server umount lustre-OST0000 complete [ 4160.995498] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4161.002172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4161.010170] Lustre: Skipped 1 previous similar message [ 4161.030367] Lustre: Skipped 1 previous similar message [ 4163.921759] Lustre: server umount lustre-MDT0000 complete [ 4166.112484] LustreError: 149813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4166.141177] LustreError: 149813:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4167.793182] LustreError: 147183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391747 with bad export cookie 12632512578638999624 [ 4167.808893] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4168.268827] Lustre: server umount lustre-MDT0001 complete [ 4180.942716] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 4184.641312] Key type lgssc unregistered [ 4185.022269] LNet: 150871:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4185.037202] LNetError: 150871:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4185.061092] LNet: Removed LNI 192.168.206.124@tcp [ 4186.153288] Key type .llcrypt unregistered [ 4186.155928] Key type ._llcrypt unregistered [ 4213.884704] Key type ._llcrypt registered [ 4213.888053] Key type .llcrypt registered [ 4214.003508] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4215.362383] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4215.494618] alg: No test for adler32 (adler32-zlib) [ 4216.617353] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 4216.933105] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 4218.599557] Key type lgssc registered [ 4220.050872] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4228.839209] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4228.852978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4230.461397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4236.636810] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4240.867024] LustreError: 152290:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4240.907607] LustreError: 152290:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4245.997973] LustreError: 152289:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4246.408433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4246.869960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4251.993419] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4259.963552] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4266.044313] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4273.503434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4273.974312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4276.021090] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4279.992361] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4287.876136] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4291.399418] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4314.552898] Lustre: lustre-MDT0001: evicting 47df80ad-f548-4aff-a3cd-9541882f1862 (at 192.168.206.24@tcp) by administrative request [ 4314.560261] Lustre: Skipped 1 previous similar message [ 4347.878704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.880718] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4347.901159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4347.920399] Lustre: Skipped 1 previous similar message [ 4349.307783] Lustre: server umount lustre-OST0000 complete [ 4354.528366] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4354.532304] LustreError: Skipped 1 previous similar message [ 4354.536487] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4354.545631] Lustre: Skipped 1 previous similar message [ 4354.548828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4358.128460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4358.130092] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4359.510679] Lustre: server umount lustre-MDT0000 complete [ 4363.224451] LustreError: 152272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787391943 with bad export cookie 5039347090584268336 [ 4363.231179] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4363.242675] LustreError: 152272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4363.542791] Lustre: server umount lustre-MDT0001 complete [ 4372.082702] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 4374.199801] Key type lgssc unregistered [ 4374.450723] LNet: 156358:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4374.458038] LNetError: 156358:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4374.482128] LNet: Removed LNI 192.168.206.124@tcp [ 4375.095876] Key type .llcrypt unregistered [ 4375.098213] Key type ._llcrypt unregistered [ 4445.993149] Key type ._llcrypt registered [ 4445.995859] Key type .llcrypt registered [ 4446.099425] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4447.346696] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4447.435711] alg: No test for adler32 (adler32-zlib) [ 4448.509348] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 4448.765057] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 4450.479199] Key type lgssc registered [ 4451.576328] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4458.245411] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4458.279969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4459.747281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4464.197693] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4469.223188] LustreError: 158845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4469.234947] LustreError: 158845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4473.225560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4473.596480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4478.143868] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4485.952816] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4491.391612] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4498.746824] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4499.189518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4505.088377] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4505.326032] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4511.989380] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4518.643429] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4524.001965] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4524.003703] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4524.013279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4524.036918] Lustre: Skipped 1 previous similar message [ 4524.051962] Lustre: Skipped 1 previous similar message [ 4525.459738] Lustre: server umount lustre-OST0000 complete [ 4530.144666] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4530.153927] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4530.171405] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4532.193845] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.195852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4532.219638] Lustre: Skipped 1 previous similar message [ 4535.325762] Lustre: server umount lustre-MDT0000 complete [ 4537.322923] LustreError: 158846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4537.341709] LustreError: 158846:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4539.530959] LustreError: 158826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787392119 with bad export cookie 11782523393368632835 [ 4539.547119] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4539.556064] LustreError: 158826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4539.960517] Lustre: server umount lustre-MDT0001 complete [ 4550.872888] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 4553.442845] Key type lgssc unregistered [ 4553.754528] LNet: 162387:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4553.767520] LNetError: 162387:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4553.788345] LNet: Removed LNI 192.168.206.124@tcp [ 4554.649871] Key type .llcrypt unregistered [ 4554.652720] Key type ._llcrypt unregistered [ 4579.373475] Key type ._llcrypt registered [ 4579.377863] Key type .llcrypt registered [ 4579.534209] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4580.839325] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4580.890598] alg: No test for adler32 (adler32-zlib) [ 4581.992617] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 4582.188125] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 4583.843733] Key type lgssc registered [ 4584.933500] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4600.921461] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:49:38 (1787392178) [ 4617.886817] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 4627.770648] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4664.122967] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4664.159965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4665.492314] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4665.525293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4665.584561] Lustre: lustre-MDT0000: new disk, initializing [ 4665.641884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4665.658566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4670.018123] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4682.270824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4682.432308] Lustre: 166656:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4682.508817] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4682.514557] Lustre: Skipped 1 previous similar message [ 4682.571831] Lustre: lustre-MDT0001: new disk, initializing [ 4682.679740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4682.869080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4682.895528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4688.514192] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4700.491372] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4705.907152] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4713.117585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4713.435522] Lustre: lustre-OST0000: new disk, initializing [ 4713.438850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4713.450485] Lustre: 168622:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4713.549188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4719.709234] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4719.721249] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4719.886886] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4720.040335] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4730.552535] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4740.580334] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4740.582706] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4740.612469] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4740.632594] Lustre: Skipped 1 previous similar message [ 4745.784154] Lustre: server umount lustre-OST0000 complete [ 4750.817826] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4750.824688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4750.840493] Lustre: Skipped 2 previous similar messages [ 4750.857272] Lustre: Skipped 1 previous similar message [ 4755.874803] Lustre: server umount lustre-MDT0000 complete [ 4755.944722] LustreError: 169147:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4755.976989] LustreError: 169147:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 4760.290932] LustreError: 166646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787392340 with bad export cookie 15977714646235089874 [ 4760.303697] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4760.308646] LustreError: 166646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4760.828361] Lustre: server umount lustre-MDT0001 complete [ 4773.801286] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 4776.654289] Key type lgssc unregistered [ 4776.955893] LNet: 170529:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4776.962085] LNetError: 170529:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4776.977229] LNet: Removed LNI 192.168.206.124@tcp [ 4777.891834] Key type .llcrypt unregistered [ 4777.894214] Key type ._llcrypt unregistered [ 4800.769370] Key type ._llcrypt registered [ 4800.772306] Key type .llcrypt registered [ 4800.888968] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 4802.162297] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4802.318982] alg: No test for adler32 (adler32-zlib) [ 4803.454989] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 4803.791044] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 4805.511169] Key type lgssc registered [ 4806.547780] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4814.950851] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4814.982981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4816.456044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4820.485953] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4826.600116] LustreError: 171945:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4826.620798] LustreError: 171945:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4829.033198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4829.380586] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4833.068513] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4839.484657] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4845.520052] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4853.489512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4853.754538] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4860.368388] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 4861.947621] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4869.480907] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4876.566948] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4876.594552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4876.618217] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4876.633868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.670907] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4877.524647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4877.548321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4877.557067] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4877.561503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.568568] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4878.507643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.467661] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4879.474269] Lustre: Skipped 1 previous similar message [ 4879.491477] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4879.500732] Lustre: Skipped 1 previous similar message [ 4879.505086] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4879.511448] Lustre: Skipped 1 previous similar message [ 4879.517338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.529471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4879.535759] Lustre: Skipped 1 previous similar message [ 4880.463965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.331145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.078692] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4882.085143] Lustre: Skipped 2 previous similar messages [ 4882.107610] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4882.113265] Lustre: Skipped 2 previous similar messages [ 4882.116751] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4882.119038] Lustre: Skipped 2 previous similar messages [ 4882.122694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.135462] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4882.139333] Lustre: Skipped 2 previous similar messages [ 4882.876757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.654840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.549097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.481151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.464811] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4886.471396] Lustre: Skipped 4 previous similar messages [ 4886.498287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4886.509459] Lustre: Skipped 4 previous similar messages [ 4886.515021] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4886.523501] Lustre: Skipped 4 previous similar messages [ 4886.528090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.545122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4886.554186] Lustre: Skipped 4 previous similar messages [ 4887.543884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.643050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.746832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.781678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.822633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.774208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.744875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.625626] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4894.631735] Lustre: Skipped 7 previous similar messages [ 4894.644541] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4894.651576] Lustre: Skipped 7 previous similar messages [ 4894.654259] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4894.658882] Lustre: Skipped 7 previous similar messages [ 4894.661291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.668761] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4894.671066] Lustre: Skipped 7 previous similar messages [ 4895.744052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.828288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.799272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.841451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.755836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.586853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.543411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.546610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.617722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.624723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.671278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.505624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.418340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.243750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.207721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.098916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.108796] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4911.115621] Lustre: Skipped 16 previous similar messages [ 4911.128712] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4911.137771] Lustre: Skipped 16 previous similar messages [ 4911.140714] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4911.151245] Lustre: Skipped 16 previous similar messages [ 4911.154775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.163252] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4911.165941] Lustre: Skipped 16 previous similar messages [ 4911.993482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.922281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.840628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.037200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.994572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.921254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.889815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.830379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.762577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.848630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.755186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.801763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.737620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.682606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.559841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.517260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.394907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.193917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.209580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.130498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.146811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.944258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.974093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.943377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.032645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.099881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.214938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.036077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.014200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.847395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.638630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.357624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.299782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.289378] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4943.295215] Lustre: Skipped 33 previous similar messages [ 4943.311657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4943.320647] Lustre: Skipped 33 previous similar messages [ 4943.324828] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4943.331284] Lustre: Skipped 33 previous similar messages [ 4943.335598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.348454] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4943.360240] Lustre: Skipped 33 previous similar messages [ 4944.264893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.102754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.950595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.832685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.720340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.543978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.423229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.361449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.426277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.448811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.369990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.284484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.236550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.125057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.971453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.774508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.767690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.661451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.609312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.522241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.318195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.124856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.886344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.633917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.480147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.466398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.353212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.129724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.232916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.327426] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4970.336789] LustreError: 171965:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4970.354704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.238571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.059800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.008969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.035389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.172053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.196792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.170466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.253438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.255560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.255583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.235745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.202336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.157077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.269338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.261898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.223665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.234506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.189522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.012666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.908871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.669049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.811713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.729917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.730764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.628117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.679151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.647252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.462982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.414844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.197100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.029355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.068124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.101675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.036864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.941914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.795795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.710910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.777656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.564437] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5007.568515] Lustre: Skipped 68 previous similar messages [ 5007.575700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5007.581540] Lustre: Skipped 68 previous similar messages [ 5007.583944] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5007.590459] Lustre: Skipped 68 previous similar messages [ 5007.592945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.600090] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5007.603640] Lustre: Skipped 68 previous similar messages [ 5008.594633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.451376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.259503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.149055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.880674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.766087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.723394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.724386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.617991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.430670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.374927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.300378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.612490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.618370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.798723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.853243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.796861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.731718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.617519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.469650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.395715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.143424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.056804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.739371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.670537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.365288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.066445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.844534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.681981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.482208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.379941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.096179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.976703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.807066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.694110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.567820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.395801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.273430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.094250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.888809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.709449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.316188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.235527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.144912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.853673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.722212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.582745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.325910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.252190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.142186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.980818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.823580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.747610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.492129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.330991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.149873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.947410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.713353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.582140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.449728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.494621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.353740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.223555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.058165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.846770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.617507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.558287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.442735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.479141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.544210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.446835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.462478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.419224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.187729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.197553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.008980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.027502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.929511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.063307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.013657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.069190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.236713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.117713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.209756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.079703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.035528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.799424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.701715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.671743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.594804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.532516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.442304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.464623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.427072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.356336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.135933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.941532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.843885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.841343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.890808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.040598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.092440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.106289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.091764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.140373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.939451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.760717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.635993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.469164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.463441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.274062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.109400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.960373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.813924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.726606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.571988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.372768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.286485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.220090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.018994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.851494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.882065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.037985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.003284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.895565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.693814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.743994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.657118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.660900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.622124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.634210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.451104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.373410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.315965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.469448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.420110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.467341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.380736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.466952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.368399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.489878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.451282] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5136.461271] Lustre: Skipped 141 previous similar messages [ 5136.480862] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5136.490789] Lustre: Skipped 141 previous similar messages [ 5136.495169] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5136.502401] Lustre: Skipped 141 previous similar messages [ 5136.506933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.525713] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5136.531276] Lustre: Skipped 141 previous similar messages [ 5137.447137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.473881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.477206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.357959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.385626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.200064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.089932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.230424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.327339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.305740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.214358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.182708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.210227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.185455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.284597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.373374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.227795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.125062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.116881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.892641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.991308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.032022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.945496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.998760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.975645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.803250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.694180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.484927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.310198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.035464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.854808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.666201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.413370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.234040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.190057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.180749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.239659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.331638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.322178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.420603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.583520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.550943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.423658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.173512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.138137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.927937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.779551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.655951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.638086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.455454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.287251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.153588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.042412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.809073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.713757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.571557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.427831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.167310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.025930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.830644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.712411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.475612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.264131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.000253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.794910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.577055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.445613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.150329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.921147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.578782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.372495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.265540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.945126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.780898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.648038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.430659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.325184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.109725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.051988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.938196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.972571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.943351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.129516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.929822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.827217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.714621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.552282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.345600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.397053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.222682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.149615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.987153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.816509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.801402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.596959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.474353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.347762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.181080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.078978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.003408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.791612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.528063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.430472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.185640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.944876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.702790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.478969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.291746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.286776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.305908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.152601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.095794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.905511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.830208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.692676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.532558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.324308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.180252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.083549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.958006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.862978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.852874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.728352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.753663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.626359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.450224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.166132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.925198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.705899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.747973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.806599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.665857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.567258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.401268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.197308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.047800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.855831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.816187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.758778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.718727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.655925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.630291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.459984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.451458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.280841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.216321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.269415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.217747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.184502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.185373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.107593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.962685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.796133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.698904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.652696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.521278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.604140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.489803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.587347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.430547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.460048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.632691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.882443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.843257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.641448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.553232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.517109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.503795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.595234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.787763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.821491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.997700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.845598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.997493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.119366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.512512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.045223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.058408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.295720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.364635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.481996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.503136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.540992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.661559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.802775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.817589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.821466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.765164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.801977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.706982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.641554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.678385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.569826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.687246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.576984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.653801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.681972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.646696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.687574] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5320.702336] LustreError: 171965:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5320.707271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.836223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.897289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.803776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.812305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.786079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.796687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.762628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.709117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.608710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.662240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.637595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.507186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.505658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.578051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.812186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.832350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.942165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.919399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.916729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.820069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.695558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.563799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.503470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.320673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.260485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.239457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.137334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.970806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.923907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.962125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.936952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.983591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.026383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.162189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.172947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.099485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.053754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.965193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.977620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.945172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.983392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.890387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.917760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.932724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.103955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.166982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.236136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.354162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.477725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.485278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.493689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.533481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.475373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.626726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.584467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.616368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.508863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.566951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.549637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.572582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.474372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.428574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.435295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.413378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.259251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.228964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.009992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.919832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.947907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.912145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.869878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.843745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.856756] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5392.865824] Lustre: Skipped 271 previous similar messages [ 5392.882879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5392.893540] Lustre: Skipped 271 previous similar messages [ 5392.897605] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5392.906138] Lustre: Skipped 271 previous similar messages [ 5392.910834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.919741] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5392.922511] Lustre: Skipped 271 previous similar messages [ 5393.759573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.716622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.717155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.643339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.596050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.481527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.322795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.382203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.452584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.403376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.311875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.087191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.110857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.972243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.885645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.718828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.693586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.516490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.527274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.455182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.479688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.611536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.580324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.415447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.375833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.347723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.227445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.085214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.977534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.053152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.952900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.806200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.793662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.814897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.735080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.568534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.397449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.273131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.065836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.914714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.698792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.655596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.543798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.519819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.360762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.109966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.944347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.679733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.573712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.396677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.153712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.974832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.963333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.809561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.674535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.500638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.339305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.102211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.931718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.751346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.439354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.268864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.066435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.827384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.610451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.422353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.227375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.175597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.092453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.819613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.688443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.637898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.458639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.454660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.450079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.486913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.425325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.321832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.368836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.269571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.466957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.407388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.416591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.597840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.040378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.179826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.258320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.101185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.120157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.252254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.275108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.205372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.532078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.814993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.824143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.746474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.782629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.833980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.808688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.903827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.994278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.873698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.897572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.780465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.822215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.641946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.760706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.716718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.690603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.703763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.748122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.746446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.718834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.666776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.626282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.630170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.665246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.511483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.439537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.405642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.254838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.295974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.213776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.161942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.978918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.879918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.739300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.622546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.377346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.471941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.228379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.192190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.110882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.015523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.783144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.715991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.575429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.450449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.277721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.145791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.981345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.040306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.923358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.729263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.659278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.592691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.564928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.572311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.697656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.774718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.918289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.836745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.110713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.142854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.086192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.335930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.808949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.936901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.075511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.172453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.314073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.542609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.544857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.730718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.601478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.544539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.473975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.494557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.456814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.509813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.480991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.391534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.262294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.158796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.120026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.108965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.236755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.300774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.321739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.329450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.480048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.557227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.498294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.455558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.475877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.522862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.690194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.646465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.710562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.746246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.812559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.756722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.834644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.786229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.699839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.879568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.858278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.942746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.036532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.167375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.439965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.365609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.456061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.107632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.927278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.800052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.965433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.883756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.906484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.732362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.593071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.489221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.623525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.430473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.592741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.652729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.733293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.602579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.388136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.407511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.264022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.170407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.154599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.161949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.992582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.007922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.091563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.232648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.191795] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5613.205988] LustreError: 171965:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5613.227633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.438196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.583068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.639348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.571984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.511347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.528168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.474709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.352402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.263260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.218425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.274773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.340137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.304624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.217870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.080696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.972868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.970488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.788620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.801693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.749968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.668706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.550691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.505320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.293600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.132727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.099418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.005937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.948472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.921360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.747544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.643666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.421897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.198312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.072810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.969982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.725637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.711327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.646337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.642316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.650161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.623602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.599089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.597506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.535262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.586585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.355247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.269359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.208545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.189955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.074379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.258760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.167436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.131819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.053629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.775552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.808691] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5665.820070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.820590] LustreError: 171965:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5666.701887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.659469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.656738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.723383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.670385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.575828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.670984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.605490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.343126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.069995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.951421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.827532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.678539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.678433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.458110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.700431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.603384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.711505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.673861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.826548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.901664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.053917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.312221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.479095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.547394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.345628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.452846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.325765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.158620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.935642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.056815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.203466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.306413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.471409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.192791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.273338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.180794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.074795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.079365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.960590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.909489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.005253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.823290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.641074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.425056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.621591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.640983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.665412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.457577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.418679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.387989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.622887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.829973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.086955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.927904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.013604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.830412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.900288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.792572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.748263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.709712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.809683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.835157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.945524] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5728.958371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.960776] LustreError: 171965:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5730.000038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.961089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.008521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.173353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.241880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.153626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.067648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.246451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.389779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.558555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.388990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.499867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.976641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.836622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.776607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.967141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.325946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.723203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.842128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.863630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.557212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.587705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.531855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.596197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.581966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.152748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.049139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.254101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.540537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.745768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.192152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.483301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.636725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.419275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.352407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.314448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.147240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.873955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.687899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.383956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.990741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.647330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.364695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.170178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.807483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.552631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.390953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.254969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.108495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.091524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.788598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.526608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.056135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.806385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.639985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.767126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.729969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.848992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.862258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.087410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.157444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.234928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.306079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.431295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.502464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.598639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.724443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.947991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.141943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.126387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.915904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.908791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.030981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.330798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.591318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.756084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.051610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.140972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.634540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.176179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.209703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.516861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.569268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.670488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.926810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.269496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.557955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.052046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.492633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.933228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.241990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.915509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.872293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.208204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.827322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.883431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.156360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.604801] LustreError: 173769:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6253.617329] LustreError: 171965:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6960.115243] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 6960.116540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6960.130703] LustreError: Skipped 1 previous similar message [ 6960.147553] Lustre: Skipped 446 previous similar messages [ 6960.161386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6964.198305] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6964.198402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6964.219172] LustreError: Skipped 5 previous similar messages [ 6966.356740] Lustre: server umount lustre-OST0000 complete [ 6974.438400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6974.454551] Lustre: Skipped 2 previous similar messages [ 6979.557688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6979.568190] Lustre: Skipped 1 previous similar message [ 6984.684854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6984.703034] Lustre: Skipped 1 previous similar message [ 6985.183192] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6985.839395] Lustre: server umount lustre-MDT0000 complete [ 6989.792400] LustreError: 171946:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6989.835455] LustreError: 171946:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6991.743329] LustreError: 171926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394571 with bad export cookie 15626453624831051858 [ 6991.760644] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6991.767312] LustreError: 171926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6992.348197] Lustre: server umount lustre-MDT0001 complete [ 7005.264741] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 7008.688451] Key type lgssc unregistered [ 7009.012585] LNet: 227229:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7009.026199] LNetError: 227229:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7009.040541] LNet: Removed LNI 192.168.206.124@tcp [ 7009.948540] Key type .llcrypt unregistered [ 7009.952576] Key type ._llcrypt unregistered [ 7034.516923] Key type ._llcrypt registered [ 7034.525479] Key type .llcrypt registered [ 7034.680066] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7035.683755] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7035.724156] alg: No test for adler32 (adler32-zlib) [ 7036.879941] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 7037.167945] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 7038.815213] Key type lgssc registered [ 7039.832392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7047.611969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7047.633177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7049.002049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7053.912353] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7058.413252] LustreError: 228625:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7058.434924] LustreError: 228625:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7062.519392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7063.006197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7067.624540] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7074.830609] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7080.380752] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7086.998695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7087.293683] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7093.485537] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7093.768939] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7101.818597] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7106.296057] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7108.131553] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7108.148163] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7108.160364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.171371] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7108.172256] LustreError: 228644:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7113.634961] Lustre: lustre-OST0000: Not available for connect from 192.168.206.24@tcp (stopping) [ 7114.212853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7114.218366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7114.222219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7116.309087] Lustre: server umount lustre-OST0000 complete [ 7128.257813] LustreError: 228644:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7151.708731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7152.274254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7154.278559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7154.304599] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7154.306755] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7159.548451] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7167.758440] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7169.303447] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7174.610164] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c210ca5000.ost_server_uuid 50 [ 7176.075683] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff98c210ca5000.ost_server_uuid in IDLE state after 0 sec [ 7181.281657] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7181.288068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7181.297069] Lustre: Skipped 1 previous similar message [ 7181.301712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7181.306238] Lustre: Skipped 1 previous similar message [ 7185.377699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7185.380814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7185.380933] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7185.419693] Lustre: Skipped 1 previous similar message [ 7186.802811] Lustre: server umount lustre-OST0000 complete [ 7190.496461] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7190.500343] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7190.512239] Lustre: Skipped 1 previous similar message [ 7190.528078] Lustre: Skipped 1 previous similar message [ 7196.681873] Lustre: server umount lustre-MDT0000 complete [ 7199.897324] LustreError: 228605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394779 with bad export cookie 7754716849247838028 [ 7199.902815] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7199.907066] LustreError: 228605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7200.156862] Lustre: server umount lustre-MDT0001 complete [ 7209.710184] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 7211.912836] Key type lgssc unregistered [ 7212.188606] LNet: 233834:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7212.200917] LNetError: 233834:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7212.226617] LNet: Removed LNI 192.168.206.124@tcp [ 7213.063709] Key type .llcrypt unregistered [ 7213.065143] Key type ._llcrypt unregistered [ 7245.308042] Key type ._llcrypt registered [ 7245.309942] Key type .llcrypt registered [ 7245.465061] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7246.246410] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7246.265451] alg: No test for adler32 (adler32-zlib) [ 7247.327340] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 7247.519393] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 7249.207445] Key type lgssc registered [ 7250.622539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7261.873123] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7261.895731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7263.400975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7267.991557] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7273.959868] LustreError: 235565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7273.984641] LustreError: 235565:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7276.672762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7276.997599] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7280.338819] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7282.976247] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7290.670863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7290.940349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7297.529579] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7307.048915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7307.335166] Lustre: lustre-OST0001: new disk, initializing [ 7307.348070] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7307.366500] Lustre: 237873:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7307.530848] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7307.687757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7307.698728] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7307.747466] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000400 [ 7308.665700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7316.623412] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7329.680489] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7337.639607] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7353.826042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7353.834445] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7353.854542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7354.851645] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7354.855292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7358.793865] Lustre: server umount lustre-MDT0000 complete [ 7359.982054] LustreError: 236294:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7362.736305] LustreError: 235546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787394942 with bad export cookie 6589327888993483973 [ 7362.751038] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7363.083880] Lustre: server umount lustre-MDT0001 complete [ 7377.452174] Lustre: server umount lustre-OST0000 complete [ 7391.749766] Lustre: server umount lustre-OST0001 complete [ 7414.347435] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7425.729288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7426.227185] LustreError: 240980:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7426.245900] LustreError: 240980:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7426.297752] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7431.072272] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7431.660422] LustreError: 240981:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7436.768720] LustreError: 240980:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7439.701731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7439.932994] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7444.248576] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7447.248480] Lustre: 242123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7453.523505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7461.089910] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7464.234971] LustreError: 242480:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7470.260084] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7470.530437] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7470.540943] Lustre: Skipped 1 previous similar message [ 7471.565856] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7471.584562] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7477.540685] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7485.630886] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7489.537622] Lustre: 243997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7501.280792] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7501.292900] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7501.309508] Lustre: Skipped 2 previous similar messages [ 7501.316826] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7501.322591] Lustre: Skipped 3 previous similar messages [ 7507.435340] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7507.439186] Lustre: Skipped 4 previous similar messages [ 7512.547093] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7512.557023] Lustre: Skipped 6 previous similar messages [ 7514.079320] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7514.335462] Lustre: server umount mylustre-MDT0000 complete [ 7517.666155] LustreError: 241714:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7517.682943] LustreError: 241714:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7517.862723] LustreError: 240955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395097 with bad export cookie 6589327888993486780 [ 7517.873514] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7517.877521] LustreError: 240955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7518.141782] Lustre: server umount mylustre-MDT0001 complete [ 7532.171313] Lustre: server umount mylustre-OST0000 complete [ 7545.374858] Lustre: server umount mylustre-OST0001 complete [ 7569.664458] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7581.297154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7581.573393] LustreError: 246306:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7581.661591] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7586.440914] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7596.236458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7602.179690] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7605.802270] Lustre: 247449:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7612.467803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7620.061394] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7626.237826] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7629.319313] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7634.953660] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7637.254594] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7646.402580] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7651.671779] Lustre: 249320:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7660.529357] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7660.555626] Lustre: Skipped 6 previous similar messages [ 7660.566522] LustreError: 246303:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7660.593409] LustreError: 246303:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 7660.813891] Lustre: server umount tfs-MDT0000 complete [ 7665.352483] LustreError: 246283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395245 with bad export cookie 6589327888993488670 [ 7665.369573] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7665.640045] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7666.048145] Lustre: server umount tfs-MDT0001 complete [ 7672.330422] Lustre: server umount tfs-OST0000 complete [ 7676.557598] Lustre: server umount tfs-OST0001 complete [ 7701.694519] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7713.758382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7714.394674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7714.405889] Lustre: Skipped 3 previous similar messages [ 7719.147731] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7729.320181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7729.637575] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7735.197202] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7738.617559] Lustre: 252710:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7746.393838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7754.193465] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7759.223809] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7764.353966] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7770.101725] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7771.983995] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7780.701405] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7784.683729] Lustre: 254582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7792.580856] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:42:49 (1787395369) [ 7811.047755] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7811.060080] Lustre: Skipped 2 previous similar messages [ 7811.078594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7811.085488] Lustre: Skipped 3 previous similar messages [ 7816.113749] Lustre: server umount lustre-MDT0000 complete [ 7816.161578] LustreError: 254130:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7816.196436] LustreError: 254130:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 16 previous similar messages [ 7820.070979] LustreError: 251543:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395399 with bad export cookie 6589327888993490336 [ 7820.079428] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7820.082898] LustreError: 251543:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7826.471819] Lustre: server umount lustre-MDT0001 complete [ 7845.343349] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7845.542544] Lustre: server umount lustre-OST0000 complete [ 7855.506895] Lustre: server umount lustre-OST0001 complete [ 7863.476622] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 7874.485926] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 7915.340842] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7915.340842] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7915.340842] [ 7915.367034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7915.593751] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7915.627711] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7915.722212] Lustre: lustre-MDT0000: new disk, initializing [ 7915.817364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7915.822608] Lustre: Skipped 3 previous similar messages [ 7915.832088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7920.330357] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7931.087913] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7931.087913] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7931.087913] [ 7931.134329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7931.291773] Lustre: 257769:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7931.366143] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7931.375103] Lustre: Skipped 1 previous similar message [ 7931.475421] Lustre: lustre-MDT0001: new disk, initializing [ 7931.601659] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7931.611399] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7936.437583] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7948.226521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7948.658204] Lustre: lustre-OST0000: new disk, initializing [ 7948.676292] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7948.684113] Lustre: 259353:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7949.875465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7949.892892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7950.012947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7955.577642] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7968.160646] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7968.445360] Lustre: lustre-OST0001: new disk, initializing [ 7968.454520] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7968.463528] Lustre: 260225:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7969.712045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7969.735557] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7969.924365] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7980.357072] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 7992.840194] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:46:10 (1787395570) [ 8007.738490] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:46:25 (1787395585) [ 8026.092259] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8026.103630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8026.113199] Lustre: Skipped 3 previous similar messages [ 8026.132275] Lustre: Skipped 5 previous similar messages [ 8028.939527] Lustre: server umount lustre-OST0000 complete [ 8034.276933] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8038.732838] Lustre: server umount lustre-MDT0000 complete [ 8042.078976] LustreError: 257761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395621 with bad export cookie 6589327888993491890 [ 8042.086064] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8042.094920] LustreError: 257761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8048.522319] Lustre: server umount lustre-MDT0001 complete [ 8061.998230] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 8076.898630] Lustre: server umount lustre-OST0001 complete [ 8084.728974] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 8100.634216] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 8139.685316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8139.964924] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8140.021542] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8140.142414] Lustre: lustre-MDT0000: new disk, initializing [ 8140.241510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8144.914578] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8157.027582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8157.148604] Lustre: 264784:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8157.228979] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8157.233608] Lustre: Skipped 1 previous similar message [ 8157.372616] Lustre: lustre-MDT0001: new disk, initializing [ 8157.557650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8157.571657] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8162.681924] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8173.132518] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8179.079063] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8186.106164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8186.378162] Lustre: 266752:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8186.469017] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8186.473759] Lustre: Skipped 5 previous similar messages [ 8188.291133] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8188.393471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8194.031760] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8205.181136] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8213.999129] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8214.001395] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8214.011559] Lustre: Skipped 4 previous similar messages [ 8214.014896] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8214.025050] LustreError: Skipped 1 previous similar message [ 8214.047520] Lustre: Skipped 7 previous similar messages [ 8218.615925] Lustre: server umount lustre-OST0000 complete [ 8224.225418] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8229.127365] Lustre: server umount lustre-MDT0000 complete [ 8229.347253] LustreError: 267583:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8229.363392] LustreError: 267583:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 8233.215656] LustreError: 265589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395813 with bad export cookie 6589327888993493360 [ 8233.220626] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8233.229118] LustreError: 265589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8233.724700] Lustre: server umount lustre-MDT0001 complete [ 8245.087890] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 8247.960466] Key type lgssc unregistered [ 8248.309731] LNet: 268517:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8248.316706] LNetError: 268517:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8248.338307] LNet: Removed LNI 192.168.206.124@tcp [ 8249.339148] Key type .llcrypt unregistered [ 8249.340592] Key type ._llcrypt unregistered [ 8277.736858] Key type ._llcrypt registered [ 8277.738634] Key type .llcrypt registered [ 8277.864345] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 8278.664707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8278.751797] alg: No test for adler32 (adler32-zlib) [ 8279.857326] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 8280.086520] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 8281.759148] Key type lgssc registered [ 8283.203348] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8290.787784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8290.815344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.207652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8296.427447] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8302.567230] LustreError: 269980:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8302.579631] LustreError: 269980:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8305.217045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8305.553109] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8309.046632] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8315.861753] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8322.377492] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8330.594873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8330.994241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8337.155164] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8338.929851] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8347.617230] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8348.656709] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8352.736931] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8352.743097] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8352.755202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8355.817463] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8355.817900] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8355.818717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8356.762955] Lustre: server umount lustre-OST0000 complete [ 8360.931490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8360.932718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8360.946796] Lustre: Skipped 1 previous similar message [ 8360.953517] Lustre: Skipped 1 previous similar message [ 8366.051097] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8366.060720] Lustre: Skipped 2 previous similar messages [ 8367.000049] Lustre: server umount lustre-MDT0000 complete [ 8371.173437] LustreError: 270707:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8371.190989] LustreError: 270707:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8371.417660] LustreError: 269959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787395951 with bad export cookie 13890945977222771256 [ 8371.426067] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8371.432740] LustreError: 269959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8371.712286] Lustre: server umount lustre-MDT0001 complete [ 8384.535458] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 8387.687241] Key type lgssc unregistered [ 8388.044614] LNet: 273566:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8388.059875] LNetError: 273566:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8389.095684] LNet: Removed LNI 192.168.206.124@tcp [ 8389.838268] Key type .llcrypt unregistered [ 8389.843225] Key type ._llcrypt unregistered [ 8406.797893] Key type ._llcrypt registered [ 8406.799827] Key type .llcrypt registered [ 8406.881832] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 8408.257326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8408.265269] alg: No test for adler32 (adler32-zlib) [ 8409.382308] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 8409.659923] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 8411.359609] Key type lgssc registered [ 8413.219797] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8424.298112] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8424.324184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8439.903953] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8445.024503] LustreError: 274761:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.124@tcp: failed processing log, type 4: rc = -110 [ 8471.711224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8477.995369] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8485.358269] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8487.618323] Lustre: server umount lustre-OST0000 complete [ 8505.247664] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 8508.280254] Key type lgssc unregistered [ 8508.583583] LNet: 276392:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8508.594550] LNetError: 276392:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8508.617943] LNet: Removed LNI 192.168.206.124@tcp [ 8509.440899] Key type .llcrypt unregistered [ 8509.444655] Key type ._llcrypt unregistered [ 8549.846795] Key type ._llcrypt registered [ 8549.849437] Key type .llcrypt registered [ 8549.988661] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 8551.024544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8551.165414] alg: No test for adler32 (adler32-zlib) [ 8552.307783] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 8552.527353] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 8554.247147] Key type lgssc registered [ 8555.318468] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8568.644401] loop0: detected capacity change from 0 to 204800000 [ 8568.850499] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8568.883245] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8569.418319] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8570.753468] loop0: detected capacity change from 0 to 204800000 [ 8571.129660] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8571.147329] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8571.698961] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8572.715931] loop0: detected capacity change from 0 to 204800000 [ 8572.924776] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8572.938045] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8573.097490] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8574.312284] loop0: detected capacity change from 0 to 204800000 [ 8574.631433] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8574.656809] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8574.843356] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8576.041693] loop0: detected capacity change from 0 to 204800000 [ 8576.096944] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8580.344525] loop0: detected capacity change from 0 to 204800000 [ 8580.372870] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8585.142618] loop0: detected capacity change from 0 to 204800000 [ 8585.187818] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8590.167082] loop0: detected capacity change from 0 to 204800000 [ 8590.210711] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8594.381524] loop0: detected capacity change from 0 to 204800000 [ 8594.454105] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8594.516406] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8594.592303] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8594.600613] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8594.611617] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8594.626040] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8594.633816] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8594.650391] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8594.666569] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8594.681652] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8594.693475] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8594.704192] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8594.717804] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8594.734361] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8594.755439] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8594.764735] Lustre: 279810:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8599.968788] Lustre: server umount MGS complete [ 8600.831138] loop0: detected capacity change from 0 to 204800000 [ 8600.914665] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8601.388130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8601.449990] Lustre: lustre-MDT0000: Aborting client recovery [ 8602.592720] loop1: detected capacity change from 0 to 204800000 [ 8602.771048] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8602.924536] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8602.944277] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8602.964753] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8602.993902] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8603.018790] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8603.029663] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8603.045821] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8603.069602] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8603.239431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8603.308973] Lustre: lustre-MDT0001: Aborting client recovery [ 8604.267287] loop2: detected capacity change from 0 to 204800000 [ 8604.436135] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8604.540981] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8604.558807] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8604.570775] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8604.578453] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8604.589223] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8604.598302] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8604.606213] Lustre: 280321:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8604.700798] Lustre: Found index 0 for lustre-OST0000, updating log [ 8604.731242] Lustre: 280340:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8604.787795] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8604.816114] Lustre: lustre-OST0000: Aborting recovery [ 8605.565420] loop3: detected capacity change from 0 to 204800000 [ 8605.680069] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8605.789614] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8605.810393] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8605.818577] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8605.827179] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8605.852224] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8605.868682] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8605.878340] Lustre: 280405:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8605.902086] Lustre: Found index 1 for lustre-OST0001, updating log [ 8605.932241] Lustre: 280411:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8606.033734] Lustre: lustre-OST0001: Aborting recovery [ 8611.108396] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8611.112969] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8611.119752] LustreError: 280186:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8611.166969] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8611.177377] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8613.159059] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8613.162715] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8614.934388] LustreError: 280153:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 470us: rc = -2 [ 8615.437242] LustreError: 280154:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 962us: rc = -2 [ 8615.461039] LustreError: 280154:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8628.708221] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8628.708949] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8628.711965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8628.743201] Lustre: Skipped 3 previous similar messages [ 8633.830352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8633.839250] Lustre: Skipped 6 previous similar messages [ 8634.332319] Lustre: server umount lustre-MDT0000 complete [ 8635.347865] LustreError: 280135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396215 with bad export cookie 10319819006202422822 [ 8635.356672] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8635.365500] LustreError: 280135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8638.948515] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8638.948776] LustreError: 280152:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8638.973966] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8638.998590] LustreError: 280152:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8639.009491] Lustre: Skipped 1 previous similar message [ 8640.992234] LustreError: 280150:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8640.992292] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8641.021502] LustreError: 280150:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8641.026562] Lustre: Skipped 1 previous similar message [ 8641.776645] Lustre: server umount lustre-MDT0001 complete [ 8648.948736] Lustre: server umount lustre-OST0000 complete [ 8656.239094] Lustre: server umount lustre-OST0001 complete [ 8665.338968] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:57:22 (1787396242) [ 8696.330410] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 8709.132187] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 8747.458796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8747.746178] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8747.776516] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8747.872505] Lustre: lustre-MDT0000: new disk, initializing [ 8747.964272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8747.970519] Lustre: Skipped 1 previous similar message [ 8747.987431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8752.392602] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8763.388492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8763.559157] Lustre: 284482:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8763.628710] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8763.638349] Lustre: Skipped 1 previous similar message [ 8763.758860] Lustre: lustre-MDT0001: new disk, initializing [ 8763.879524] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8763.916490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8763.926910] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8768.633680] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8778.983902] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8785.101833] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8792.122636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8792.442440] Lustre: lustre-OST0000: new disk, initializing [ 8792.445255] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8792.454906] Lustre: 286449:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8792.567281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8798.656577] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8801.840185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8801.847832] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8801.967783] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8808.269280] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8814.030930] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8827.052759] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8835.037162] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8843.202834] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8888.803519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8888.821085] Lustre: Skipped 1 previous similar message [ 8888.826242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8888.832350] Lustre: Skipped 1 previous similar message [ 8891.684903] Lustre: server umount lustre-OST0000 complete [ 8896.994282] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8897.000520] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8897.007202] Lustre: Skipped 1 previous similar message [ 8897.010830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8910.303287] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8910.737289] Lustre: server umount lustre-MDT0000 complete [ 8913.376666] LustreError: 284489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8913.398358] LustreError: 284489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8915.010872] LustreError: 287995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396494 with bad export cookie 10319819006202425531 [ 8915.012020] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8915.025404] LustreError: 287995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8915.474820] Lustre: server umount lustre-MDT0001 complete [ 8926.226632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8931.486844] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8936.928597] Lustre: server umount MGS complete [ 8949.880155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8950.446356] LustreError: 290435:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8950.515559] LustreError: 290448:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8950.680178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8955.879467] LustreError: 290436:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8957.616269] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8966.125927] LustreError: 290435:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8966.150074] LustreError: 290435:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8967.675206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8972.284784] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 8979.609983] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8985.581836] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8992.203367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8992.502214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8992.508826] Lustre: Skipped 1 previous similar message [ 8999.236589] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9006.328491] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9031.648492] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9031.655558] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9031.662501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9031.662508] Lustre: Skipped 6 previous similar messages [ 9031.690243] Lustre: Skipped 2 previous similar messages [ 9037.410605] Lustre: server umount lustre-OST0000 complete [ 9041.891109] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9047.750383] Lustre: server umount lustre-MDT0000 complete [ 9051.788858] LustreError: 290417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396631 with bad export cookie 10319819006202427470 [ 9051.791134] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9051.809893] LustreError: 290417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9052.333255] Lustre: server umount lustre-MDT0001 complete [ 9064.852084] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 9067.925848] Key type lgssc unregistered [ 9068.268194] LNet: 294380:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9068.276632] LNetError: 294380:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9068.288961] LNet: Removed LNI 192.168.206.124@tcp [ 9069.346357] Key type .llcrypt unregistered [ 9069.347922] Key type ._llcrypt unregistered [ 9108.089546] Key type ._llcrypt registered [ 9108.092681] Key type .llcrypt registered [ 9108.210124] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 9122.352693] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 9123.399773] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9123.639158] alg: No test for adler32 (adler32-zlib) [ 9124.628947] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 9124.880308] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 9126.543273] Key type lgssc registered [ 9128.194534] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9167.174529] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9167.222550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9168.565896] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9168.612035] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9168.706912] Lustre: lustre-MDT0000: new disk, initializing [ 9168.837927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9168.863710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9174.292744] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9184.836618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9184.996668] Lustre: 298337:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9185.078780] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9185.084995] Lustre: Skipped 1 previous similar message [ 9185.189800] Lustre: lustre-MDT0001: new disk, initializing [ 9185.272778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9185.319861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9185.328050] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9189.474659] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9201.034721] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9208.096280] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9215.966926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9216.305929] Lustre: lustre-OST0000: new disk, initializing [ 9216.312183] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9216.316249] Lustre: 300307:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9216.395468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9222.492382] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9226.298235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9226.308874] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9226.401986] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9231.497519] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9239.095122] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9251.866484] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9259.391650] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9267.175736] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9313.766057] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9313.766715] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9313.781213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9313.786075] LustreError: Skipped 1 previous similar message [ 9313.822463] Lustre: Skipped 1 previous similar message [ 9316.835958] Lustre: server umount lustre-OST0000 complete [ 9323.498237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9323.510948] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9323.539688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9323.549911] Lustre: Skipped 1 previous similar message [ 9327.193558] Lustre: server umount lustre-MDT0000 complete [ 9331.004392] LustreError: 298328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787396910 with bad export cookie 1545878091059994569 [ 9331.010500] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9331.019142] LustreError: 298328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9331.439577] Lustre: server umount lustre-MDT0001 complete [ 9340.565384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9344.849721] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9349.633517] Lustre: server umount MGS complete [ 9361.247774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9361.778188] LustreError: 304278:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9361.797765] LustreError: 304278:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9361.811875] LustreError: 304291:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9361.878139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9366.454364] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9367.011241] LustreError: 304279:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9371.108552] LustreError: 304278:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9375.143345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9375.505801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9380.455853] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9388.501838] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9394.737630] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9402.666378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9403.352184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9410.319585] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9417.926555] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9446.381045] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9446.382675] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9446.403451] Lustre: Skipped 1 previous similar message [ 9446.410750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9446.416282] Lustre: Skipped 3 previous similar messages [ 9446.947921] Lustre: server umount lustre-OST0000 complete [ 9451.490301] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9451.511270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9451.524576] Lustre: Skipped 2 previous similar messages [ 9456.610585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9456.619253] Lustre: Skipped 2 previous similar messages [ 9465.311872] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9465.854105] Lustre: server umount lustre-MDT0000 complete [ 9466.852206] LustreError: 304274:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9466.875611] LustreError: 304274:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9469.563088] LustreError: 305015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397049 with bad export cookie 1545878091059996522 [ 9469.575172] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9469.585767] LustreError: 305015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9470.174787] Lustre: server umount lustre-MDT0001 complete [ 9481.729795] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [ 9484.440443] Key type lgssc unregistered [ 9484.795877] LNet: 308224:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9484.814248] LNetError: 308224:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9485.866871] LNet: Removed LNI 192.168.206.124@tcp [ 9486.711683] Key type .llcrypt unregistered [ 9486.715369] Key type ._llcrypt unregistered [ 9509.990380] Key type ._llcrypt registered [ 9509.993093] Key type .llcrypt registered [ 9510.114218] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [ 9523.203639] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 9524.132969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9524.150500] alg: No test for adler32 (adler32-zlib) [ 9525.346778] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [ 9525.669851] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [ 9527.415435] Key type lgssc registered [ 9528.959779] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9552.468488] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [ 9569.124397] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9569.165318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9570.623511] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9570.666654] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9570.796680] Lustre: lustre-MDT0000: new disk, initializing [ 9570.937072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9570.962076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9576.109105] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9586.936514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9587.173811] Lustre: lustre-OST0000: new disk, initializing [ 9587.177244] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9587.180663] Lustre: Skipped 1 previous similar message [ 9587.184790] Lustre: 312256:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9587.257900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9595.450840] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [ 9596.036928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9596.050441] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9596.098733] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10496.995172] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10497.021302] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10500.784349] Lustre: server umount lustre-MDT0000 complete [10504.337959] LustreError: 311441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398084 with bad export cookie 4036644606697192791 [10504.352105] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10504.357666] LustreError: 311441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10504.515668] Lustre: server umount lustre-OST0000 complete [10533.768883] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [10544.763363] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [10581.713149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.930395] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10581.973439] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10582.050453] Lustre: lustre-MDT0000: new disk, initializing [10582.116578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10582.150856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10586.248726] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10595.549236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10595.710481] Lustre: 316606:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10595.780394] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10595.786952] Lustre: Skipped 1 previous similar message [10595.846666] Lustre: lustre-MDT0001: new disk, initializing [10595.906919] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10595.947504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10595.962092] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10600.105326] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10609.637143] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10614.631468] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10620.793671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10621.125919] Lustre: lustre-OST0000: new disk, initializing [10621.128791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10621.135995] Lustre: 318573:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10621.273840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10627.811532] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10628.152292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10628.164351] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10628.293318] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10636.116764] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10641.517993] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10641.735812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10646.823257] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10647.024723] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10649.063899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10649.068378] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10649.095977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10654.186138] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10654.190194] Lustre: Skipped 1 previous similar message [10659.301095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10659.307268] Lustre: Skipped 2 previous similar messages [10663.391120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10663.520765] Lustre: server umount lustre-OST0000 complete [10667.492066] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10667.507547] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10667.520222] Lustre: Skipped 1 previous similar message [10667.525506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10669.546540] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10673.642324] Lustre: server umount lustre-MDT0000 complete [10677.461924] LustreError: 316598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398257 with bad export cookie 4036644606700133610 [10677.472491] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10677.478567] LustreError: 316598:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10677.919244] Lustre: server umount lustre-MDT0001 complete [10681.413698] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10682.806424] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:31:00 (1787398260) [10689.472677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10689.762362] LustreError: 320885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10689.773808] LustreError: 320885:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10689.809949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10693.759883] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10695.138326] LustreError: 320886:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10699.238612] LustreError: 320885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10702.783172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10703.302409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10708.435070] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10716.786374] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10722.546932] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10729.485195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10729.830899] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10736.475725] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10743.639313] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10749.983631] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10750.105794] Lustre: lustre-OST0001: new disk, initializing [10750.115550] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10750.127215] Lustre: 323721:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10750.229612] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10755.924583] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10756.202536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10756.219546] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10756.259174] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10757.287992] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10757.298637] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10758.373745] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10758.385074] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10760.428950] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10760.446954] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10764.585130] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10764.600604] LustreError: 324102:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10766.019818] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10766.261339] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10772.480656] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c213cca800.ost_server_uuid 50 [10772.774642] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10772.788491] LustreError: 324095:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10773.988443] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff98c213cca800.ost_server_uuid in FULL state after 0 sec [10779.696416] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98c213cca800.ost_server_uuid 50 [10781.236649] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff98c213cca800.ost_server_uuid in FULL state after 0 sec [10783.101092] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10796.457669] Lustre: lustre-OST0001: Not available for connect from 192.168.206.24@tcp (stopping) [10796.463078] Lustre: Skipped 2 previous similar messages [10797.546757] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10797.573415] Lustre: Skipped 1 previous similar message [10801.811582] Lustre: server umount lustre-OST0001 complete [10802.671355] LustreError: 322708:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10802.710719] LustreError: 322708:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10807.779090] LustreError: 325288:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10807.801173] LustreError: 325288:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10809.824446] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10809.830098] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10811.881286] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10812.896142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10812.902152] Lustre: Skipped 6 previous similar messages [10814.536330] Lustre: server umount lustre-OST0000 complete [10821.089077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10824.872272] Lustre: server umount lustre-MDT0000 complete [10828.259916] LustreError: 320886:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10828.277511] LustreError: 320886:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10829.370170] LustreError: 320867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398409 with bad export cookie 4036644606700134492 [10829.393163] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10829.993870] Lustre: server umount lustre-MDT0001 complete [10843.343771] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [10847.422388] Key type lgssc unregistered [10847.881599] LNet: 326412:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10847.894941] LNetError: 326412:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10848.934735] LNet: Removed LNI 192.168.206.124@tcp [10849.910453] Key type .llcrypt unregistered [10849.913834] Key type ._llcrypt unregistered [10878.096042] Key type ._llcrypt registered [10878.101352] Key type .llcrypt registered [10878.215953] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [10879.400483] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10879.420796] alg: No test for adler32 (adler32-zlib) [10880.538860] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [10880.822053] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [10882.575453] Key type lgssc registered [10883.638855] Lustre: Echo OBD driver; http://www.lustre.org/ [10890.941909] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10890.957521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10892.321102] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10896.071137] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10902.307643] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10902.497769] LustreError: 327835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10902.515823] LustreError: 327835:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10907.627574] LustreError: 327836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10908.623341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10909.119245] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10912.706977] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10922.418889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10922.691756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10928.198345] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10933.988251] LustreError: 329516:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10935.094950] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10938.093732] LustreError: 329516:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10938.103099] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10938.116926] LustreError: 329516:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10941.016593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10941.245472] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10946.551223] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [10946.556979] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10953.874896] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10960.396969] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10960.526936] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11069.424567] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11069.432887] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11069.443223] LustreError: Skipped 1 previous similar message [11069.461744] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11069.469086] Lustre: Skipped 1 previous similar message [11071.466031] Lustre: server umount lustre-OST0001 complete [11074.551391] LustreError: 329517:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11074.578218] LustreError: 329517:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11079.650063] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11079.665768] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11079.668532] Lustre: Skipped 2 previous similar messages [11079.688144] Lustre: Skipped 1 previous similar message [11084.483558] Lustre: server umount lustre-OST0000 complete [11088.356958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11088.368390] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11088.396485] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11094.470203] Lustre: server umount lustre-MDT0000 complete [11095.011589] LustreError: 327830:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11095.022765] LustreError: 327830:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11097.995375] LustreError: 331519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398677 with bad export cookie 2288081745871065724 [11098.001531] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11098.007712] LustreError: 331519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11098.394186] Lustre: server umount lustre-MDT0001 complete [11108.478595] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [11110.899813] Key type lgssc unregistered [11111.261447] LNet: 332777:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11111.290227] LNetError: 332777:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11111.305718] LNet: Removed LNI 192.168.206.124@tcp [11112.024738] Key type .llcrypt unregistered [11112.030951] Key type ._llcrypt unregistered [11160.037546] Key type ._llcrypt registered [11160.042876] Key type .llcrypt registered [11160.136215] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [11161.304726] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11161.316601] alg: No test for adler32 (adler32-zlib) [11162.253546] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [11162.534147] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [11164.231237] Key type lgssc registered [11165.171338] Lustre: Echo OBD driver; http://www.lustre.org/ [11173.781545] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11173.839479] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11175.366104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11179.508792] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11185.643309] LustreError: 335071:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.658546] LustreError: 335071:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11188.275757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11188.711836] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11193.032204] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11195.993486] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11202.707383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11203.156873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11210.232062] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11213.412392] LustreError: 336558:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11218.930368] LustreError: 336560:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11220.585225] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11220.922810] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11222.049204] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11222.050248] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11227.404378] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11237.072178] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11244.051753] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11257.825895] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11257.832271] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11257.837077] Lustre: Skipped 2 previous similar messages [11257.859583] Lustre: Skipped 3 previous similar messages [11262.625940] Lustre: server umount lustre-MDT0000 complete [11262.947681] LustreError: 335071:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11262.972778] LustreError: 335071:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11266.517983] LustreError: 335051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398846 with bad export cookie 5212248676561787421 [11266.524681] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11266.535837] LustreError: 335051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11266.851767] Lustre: server umount lustre-MDT0001 complete [11280.040614] Lustre: server umount lustre-OST0000 complete [11283.424865] Lustre: 334333:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787398847/real 1787398847] req@ffff9e4d2aa3df80 x1874223423355904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787398863 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11283.471308] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11283.486398] Lustre: Skipped 1 previous similar message [11283.693463] Lustre: server umount lustre-OST0001 complete [11298.277920] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [11307.285279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11307.561231] LustreError: 340332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11307.632359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11311.445907] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11317.221127] LustreError: 340332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11317.237418] LustreError: 340332:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11319.351679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11319.774289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11324.178493] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11326.889682] Lustre: 341473:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11333.109389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11340.083846] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11344.748202] LustreError: 341827:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11349.222255] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11349.435218] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11349.445553] Lustre: Skipped 1 previous similar message [11353.596914] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11353.608875] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11355.756884] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11363.785214] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11367.685498] Lustre: 343346:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11374.054849] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11374.062909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11374.082212] Lustre: Skipped 2 previous similar messages [11374.103931] Lustre: Skipped 3 previous similar messages [11378.385325] Lustre: server umount lustre-MDT0000 complete [11379.169467] LustreError: 340333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11379.194911] LustreError: 340333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11381.809414] LustreError: 340314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787398961 with bad export cookie 5212248676561789122 [11381.810901] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11381.824100] LustreError: 340314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11382.166865] Lustre: server umount lustre-MDT0001 complete [11395.401220] Lustre: server umount lustre-OST0000 complete [11408.445148] Lustre: server umount lustre-OST0001 complete [11411.959781] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11413.438531] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11414.956968] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:43:12 (1787398992) [11438.924419] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [11448.078767] loop0: detected capacity change from 0 to 1099511627776 [11448.503269] SGI XFS with ACLs, security attributes, quota, no debug enabled [11448.508336] *** VALIDATE xfs *** [11448.516445] XFS (loop0): Mounting V5 Filesystem [11448.532741] XFS (loop0): Ending clean mount [11450.022119] loop1: detected capacity change from 0 to 18691697672192 [11561.882696] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [11571.910546] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [11613.461717] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11613.693927] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11613.721902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11613.801552] Lustre: lustre-MDT0000: new disk, initializing [11613.937379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11613.952403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11619.364147] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11630.388101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11630.539253] Lustre: 348486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11630.692624] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11630.703714] Lustre: Skipped 1 previous similar message [11630.801690] Lustre: lustre-MDT0001: new disk, initializing [11630.940996] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11630.950462] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11634.956294] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11645.496344] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11651.629784] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11658.454086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11658.727794] Lustre: lustre-OST0000: new disk, initializing [11658.732192] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11658.744774] Lustre: 350452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11660.203608] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11660.210987] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11660.370747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11665.130971] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11675.166683] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11679.769792] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11679.996459] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11684.472273] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11684.691983] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11690.976396] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11690.979962] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11690.992113] LustreError: Skipped 1 previous similar message [11690.992797] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11691.028596] Lustre: Skipped 2 previous similar messages [11692.993878] Lustre: server umount lustre-OST0000 complete [11697.635646] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11697.647683] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11697.661102] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11697.676950] Lustre: Skipped 1 previous similar message [11703.241172] Lustre: server umount lustre-MDT0000 complete [11704.292411] LustreError: 348497:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11704.312819] LustreError: 348497:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11707.482523] LustreError: 348477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399287 with bad export cookie 5212248676561790676 [11707.503909] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11707.940933] Lustre: server umount lustre-MDT0001 complete [11717.044964] XFS (loop0): Unmounting Filesystem [11722.495966] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:48:19 (1787399299) [11729.139234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11729.693168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11729.708965] Lustre: Skipped 2 previous similar messages [11734.065587] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11742.892560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11747.480471] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11755.032646] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11762.507655] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11771.118315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11778.447066] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11786.675845] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11800.544768] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11800.554080] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11800.574743] Lustre: Skipped 1 previous similar message [11800.580296] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11800.584303] Lustre: Skipped 2 previous similar messages [11804.642627] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11805.912557] Lustre: server umount lustre-OST0000 complete [11814.893098] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11814.894256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11814.903788] Lustre: Skipped 3 previous similar messages [11816.556123] Lustre: server umount lustre-MDT0000 complete [11820.538801] LustreError: 352992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787399400 with bad export cookie 5212248676561791558 [11820.559220] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11820.559950] LustreError: 352992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [11821.024410] Lustre: server umount lustre-MDT0001 complete [11833.651224] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [11836.552994] Key type lgssc unregistered [11836.950677] LNet: 356643:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11836.955897] LNetError: 356643:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11836.977218] LNet: Removed LNI 192.168.206.124@tcp [11837.954235] Key type .llcrypt unregistered [11837.957760] Key type ._llcrypt unregistered [11871.363897] Key type ._llcrypt registered [11871.368152] Key type .llcrypt registered [11871.559888] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [11872.568001] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11872.715587] alg: No test for adler32 (adler32-zlib) [11873.872478] Lustre: Lustre: Build Version: 2.17.54_111_ga5eb910 [11874.098334] LNet: Added LNI 192.168.206.124@tcp [8/256/0/180] [11875.847201] Key type lgssc registered [11876.899342] Lustre: Echo OBD driver; http://www.lustre.org/ [11884.522925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11884.534115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11885.905758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11889.952555] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11895.281994] LustreError: 358215:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11895.295782] LustreError: 358215:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11900.017649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11900.387044] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11900.743392] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11906.108772] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11914.455895] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11920.844082] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11928.026492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11928.533754] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11935.632371] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11943.653855] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11955.611552] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.24@tcp (stopping) [11956.204528] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11960.162854] Lustre: server umount lustre-MDT0001 complete [11960.696549] LustreError: 358211:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11965.821417] LustreError: 358210:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11965.853501] LustreError: 358210:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11970.948626] LustreError: 358210:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11970.971407] LustreError: 358210:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11972.362977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11972.460701] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11972.465159] Lustre: Found index 1 for lustre-MDT0001, updating log [11972.499449] Lustre: 358202:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11972.791625] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11978.224427] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11978.328888] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [11978.765691] LustreError: 361681:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11978.781665] LustreError: 361681:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11978.789802] LustreError: 358206:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11978.871942] LustreError: 361688:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.124@tcp: cfg command failed: rc = -17 [11978.890871] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11978.890871] [11988.449893] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11988.457711] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11988.464158] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11988.464167] Lustre: Skipped 2 previous similar messages [11988.506234] Lustre: Skipped 2 previous similar messages [11991.451206] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.24@tcp (stopping) [11991.460052] Lustre: Skipped 1 previous similar message [11994.494486] Lustre: server umount lustre-MDT0001 complete [11996.537984] LustreError: 358211:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11996.561858] LustreError: 358211:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12005.380867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12005.501390] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12005.506213] Lustre: Found index 1 for lustre-MDT0001, updating log [12005.556817] Lustre: 358202:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12005.889965] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12010.990084] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12011.466343] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [12012.045447] LustreError: 362764:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12012.062524] LustreError: 362764:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12012.070509] LustreError: 358206:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12012.076202] LustreError: 358206:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12012.197860] LustreError: 362771:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12012.207852] LustreError: 362771:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12012.216105] LustreError: 362771:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.124@tcp: cfg command failed: rc = -17 [12012.231568] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.124@tcp [12012.231568] [12318.178552] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12318.181450] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12318.183463] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12318.183470] Lustre: Skipped 2 previous similar messages [12318.210831] Lustre: Skipped 1 previous similar message [12323.140744] Lustre: server umount lustre-MDT0001 complete [12323.302759] LustreError: 358216:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12323.334077] LustreError: 358216:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [12333.665283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12333.754270] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12333.759628] Lustre: Found index 1 for lustre-MDT0001, updating log [12333.802992] Lustre: 358204:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12334.382766] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12338.759780] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [12339.188983] LustreError: 371176:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12339.198700] LustreError: 371176:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12339.212396] LustreError: 358206:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12339.218868] LustreError: 358206:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12339.323566] LustreError: 371182:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12339.371122] LustreError: 371182:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.124@tcp: cfg command failed: rc = -114 [12339.382195] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12339.382195] [12339.693572] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12662.250397] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12662.251823] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12662.252881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12662.252887] Lustre: Skipped 2 previous similar messages [12662.306632] Lustre: Skipped 2 previous similar messages [12667.620632] Lustre: server umount lustre-MDT0000 complete [12671.993031] LustreError: 358195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400251 with bad export cookie 18329174307576963932 [12672.004775] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12672.013921] LustreError: 358195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12672.482099] LustreError: 366176:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12672.487666] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12672.503069] LustreError: 366176:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [12678.588613] Lustre: server umount lustre-MDT0001 complete [12683.229680] Lustre: server umount lustre-OST0000 complete [12694.397155] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [12705.311268] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [12749.475563] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:05:26 (1787400326) [12758.098377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12758.398961] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12758.456270] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12758.597406] Lustre: lustre-MDT0000: new disk, initializing [12758.691980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12758.715492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12763.817763] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [12775.521038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12775.728392] Lustre: 381909:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12775.824827] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12775.836561] Lustre: Skipped 1 previous similar message [12775.999154] Lustre: lustre-MDT0001: new disk, initializing [12776.180427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12776.267496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12776.284824] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12781.331676] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [12792.200047] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12798.748718] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12806.916909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12807.240992] Lustre: lustre-OST0000: new disk, initializing [12807.244826] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12807.254925] Lustre: 383882:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12807.360796] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12814.411567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12814.430123] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12814.598583] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12814.703432] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_default_debug -1 all [12825.511751] Lustre: DEBUG MARKER: oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12834.665152] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12834.678301] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12834.698215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12834.708785] Lustre: Skipped 7 previous similar messages [12835.102217] Lustre: server umount lustre-MDT0000 complete [12835.298996] LustreError: 381922:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12835.327909] LustreError: 381922:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12839.600424] LustreError: 382707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787400419 with bad export cookie 18329174307576965885 [12839.605720] LustreError: MGC192.168.206.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12839.615854] LustreError: 382720:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12839.618355] LustreError: 382707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12840.038826] Lustre: server umount lustre-MDT0001 complete [12855.327034] Lustre: server umount lustre-OST0000 complete [12889.734122] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing set_hostid [12899.938930] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing load_modules_local [12937.162497] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12668 sec ========= 08:08:34 (1787400514) [12939.232301] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:08:36 (1787400516) === [12943.037871] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:08:40 (1787400520) === [12987.264038] Lustre: DEBUG MARKER: oleg624-server.virtnet: executing unload_modules_local [12990.544432] Key type lgssc unregistered [12990.859664] LNet: 390248:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12990.865824] LNetError: 390248:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12990.877571] LNet: Removed LNI 192.168.206.124@tcp [12991.997164] Key type .llcrypt unregistered [12991.999844] Key type ._llcrypt unregistered