[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 703109468 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002333] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004012] kvm-guest: setup PV IPIs [ 0.007532] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010140] LSM: Security Framework initializing [ 0.011050] Yama: becoming mindful. [ 0.012038] SELinux: Initializing. [ 0.013080] *** VALIDATE selinux *** [ 0.021294] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025828] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027158] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028111] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029122] *** VALIDATE tmpfs *** [ 0.030464] *** VALIDATE proc *** [ 0.032126] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034264] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035153] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037028] Spectre V2 : User space: Vulnerable [ 0.038010] Speculative Store Bypass: Vulnerable [ 0.041019] debug: unmapping init [mem 0xffffffffb9c59000-0xffffffffb9c60fff] [ 0.043177] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044727] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045024] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047014] ... generic registers: 4 [ 0.048012] ... value mask: 0000ffffffffffff [ 0.049015] ... max period: 00007fffffffffff [ 0.050016] ... fixed-purpose events: 3 [ 0.051013] ... event mask: 000000070000000f [ 0.052326] rcu: Hierarchical SRCU implementation. [ 0.054523] smp: Bringing up secondary CPUs ... [ 0.055617] x86: Booting SMP configuration: [ 0.056022] .... node #0, CPUs: #1 #2 #3 [ 0.059173] smp: Brought up 1 node, 4 CPUs [ 0.061013] smpboot: Max logical packages: 1 [ 0.062019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.120623] node 0 deferred pages initialised in 56ms [ 0.124203] devtmpfs: initialized [ 0.125285] x86/mm: Memory block size: 128MB [ 0.128859] gcov: version magic: 0x41383552 [ 0.131405] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.135118] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.137317] pinctrl core: initialized pinctrl subsystem [ 0.139212] [ 0.139605] ************************************************************* [ 0.141010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.143008] ** ** [ 0.144008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.146009] ** ** [ 0.147007] ** This means that this kernel is built to expose internal ** [ 0.149009] ** IOMMU data structures, which may compromise security on ** [ 0.150006] ** your system. ** [ 0.152008] ** ** [ 0.153008] ** If you see this message and you are not debugging the ** [ 0.154009] ** kernel, report this immediately to your vendor! ** [ 0.156007] ** ** [ 0.157009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159011] ************************************************************* [ 0.160613] NET: Registered protocol family 16 [ 0.162425] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.165063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.168067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.171454] cpuidle: using governor menu [ 0.173559] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.176084] PCI: Using configuration type 1 for base access [ 0.178142] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.187050] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.189036] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.193090] cryptd: max_cpu_qlen set to 1000 [ 0.195216] ACPI: Added _OSI(Module Device) [ 0.197016] ACPI: Added _OSI(Processor Device) [ 0.198011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.200014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.204000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.210382] ACPI: Interpreter enabled [ 0.211000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.213016] ACPI: Using IOAPIC for interrupt routing [ 0.215124] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.218273] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.230597] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.232039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.234022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.237085] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.241402] acpiphp: Slot [2] registered [ 0.243130] acpiphp: Slot [5] registered [ 0.245154] acpiphp: Slot [6] registered [ 0.246105] acpiphp: Slot [7] registered [ 0.247126] acpiphp: Slot [8] registered [ 0.248095] acpiphp: Slot [9] registered [ 0.250121] acpiphp: Slot [10] registered [ 0.251113] acpiphp: Slot [3] registered [ 0.253082] acpiphp: Slot [4] registered [ 0.254070] acpiphp: Slot [11] registered [ 0.255097] acpiphp: Slot [12] registered [ 0.257166] acpiphp: Slot [13] registered [ 0.258133] acpiphp: Slot [14] registered [ 0.260115] acpiphp: Slot [15] registered [ 0.261106] acpiphp: Slot [16] registered [ 0.263107] acpiphp: Slot [17] registered [ 0.264115] acpiphp: Slot [18] registered [ 0.266105] acpiphp: Slot [19] registered [ 0.267104] acpiphp: Slot [20] registered [ 0.269130] acpiphp: Slot [21] registered [ 0.271106] acpiphp: Slot [22] registered [ 0.272134] acpiphp: Slot [23] registered [ 0.274110] acpiphp: Slot [24] registered [ 0.275113] acpiphp: Slot [25] registered [ 0.277107] acpiphp: Slot [26] registered [ 0.278131] acpiphp: Slot [27] registered [ 0.279112] acpiphp: Slot [28] registered [ 0.281109] acpiphp: Slot [29] registered [ 0.283116] acpiphp: Slot [30] registered [ 0.284112] acpiphp: Slot [31] registered [ 0.286111] PCI host bridge to bus 0000:00 [ 0.287019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.290025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.293026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.296029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.299027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.301025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.303229] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.305890] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.309303] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.318761] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.323812] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.326021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.328018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.331017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.334413] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.336607] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.338025] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.340751] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.345015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.357014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.361016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.365840] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.371018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.382019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.392017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.400631] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.409016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.415017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.436018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.450504] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.456020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.464017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.479021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.489000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.499016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.509018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.525019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.535570] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.541026] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.546020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.558023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.567000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.572021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.580024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.593024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.605104] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.607378] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.610373] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.612380] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.615222] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.620128] iommu: Default domain type: Passthrough [ 0.622560] SCSI subsystem initialized [ 0.624178] ACPI: bus type USB registered [ 0.625127] usbcore: registered new interface driver usbfs [ 0.629135] usbcore: registered new interface driver hub [ 0.631151] usbcore: registered new device driver usb [ 0.633217] pps_core: LinuxPPS API ver. 1 registered [ 0.634013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.637103] PTP clock support registered [ 0.639113] EDAC MC: Ver: 3.0.0 [ 0.641165] PCI: Using ACPI for IRQ routing [ 0.642873] NetLabel: Initializing [ 0.644015] NetLabel: domain hash size = 128 [ 0.646013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.648082] NetLabel: unlabeled traffic allowed by default [ 0.650154] vgaarb: loaded [ 0.651249] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.654017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.662000] clocksource: Switched to clocksource kvm-clock [ 0.778588] VFS: Disk quotas dquot_6.6.0 [ 0.781093] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.783649] *** VALIDATE ramfs *** [ 0.785045] *** VALIDATE hugetlbfs *** [ 0.787263] pnp: PnP ACPI init [ 0.789485] pnp: PnP ACPI: found 6 devices [ 0.806969] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.809866] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.811373] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.812961] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.814544] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.816133] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.818027] NET: Registered protocol family 2 [ 0.819713] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.823375] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.827000] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.832095] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.835351] TCP: Hash tables configured (established 65536 bind 65536) [ 0.837744] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.842048] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.844793] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.847850] NET: Registered protocol family 1 [ 0.850803] RPC: Registered named UNIX socket transport module. [ 0.853546] RPC: Registered udp transport module. [ 0.855015] RPC: Registered tcp transport module. [ 0.856379] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.858467] NET: Registered protocol family 44 [ 0.859669] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.861518] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.864051] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.866438] PCI: CLS 0 bytes, default 64 [ 0.867749] Unpacking initramfs... [ 2.168842] debug: unmapping init [mem 0xffff90c13cc54000-0xffff90c13ffbffff] [ 2.173025] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.175309] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.177513] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.680818] Initialise system trusted keyrings [ 2.682635] Key type blacklist registered [ 2.684499] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.693239] zbud: loaded [ 2.696440] *** VALIDATE nfs *** [ 2.697813] *** VALIDATE nfs4 *** [ 2.699456] pstore: using deflate compression [ 2.702682] Platform Keyring initialized [ 2.814509] NET: Registered protocol family 38 [ 2.816488] Key type asymmetric registered [ 2.818074] Asymmetric key parser 'x509' registered [ 2.819825] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.822720] io scheduler mq-deadline registered [ 2.823735] io scheduler kyber registered [ 2.824720] io scheduler bfq registered [ 2.826653] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.830185] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.832587] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.835576] ACPI: Power Button [PWRF] [ 2.841172] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.847678] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.861651] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.868242] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.887170] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.913758] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.944065] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.949095] Non-volatile memory driver v1.3 [ 2.950953] Linux agpgart interface v0.103 [ 2.986369] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.989489] vda: detected capacity change from 0 to 74702848 [ 3.009529] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.012445] vdb: detected capacity change from 0 to 1073741824 [ 3.027130] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.030398] vdc: detected capacity change from 0 to 2621440000 [ 3.045241] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.048339] vdd: detected capacity change from 0 to 2621440000 [ 3.062026] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.064599] vde: detected capacity change from 0 to 4294967296 [ 3.079118] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.081541] vdf: detected capacity change from 0 to 4294967296 [ 3.089997] libphy: Fixed MDIO Bus: probed [ 3.096762] usbcore: registered new interface driver usbserial_generic [ 3.098561] usbserial: USB Serial support registered for generic [ 3.100796] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.105194] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.106921] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.109252] mousedev: PS/2 mouse device common for all mice [ 3.112318] rtc_cmos 00:05: RTC can wake from S4 [ 3.114359] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.119465] rtc_cmos 00:05: registered as rtc0 [ 3.124333] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.127545] intel_pstate: CPU model not supported [ 3.130456] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.132412] hid: raw HID events driver (C) Jiri Kosina [ 3.135777] usbcore: registered new interface driver usbhid [ 3.138221] usbhid: USB HID core driver [ 3.138581] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.139424] drop_monitor: Initializing network drop monitor service [ 3.143729] Initializing XFRM netlink socket [ 3.145597] NET: Registered protocol family 10 [ 3.147517] Segment Routing with IPv6 [ 3.148861] NET: Registered protocol family 17 [ 3.150683] mpls_gso: MPLS GSO support [ 3.156103] RAS: Correctable Errors collector initialized. [ 3.158191] AVX version of gcm_enc/dec engaged. [ 3.159500] AES CTR mode by8 optimization enabled [ 3.242917] sched_clock: Marking stable (3242880383, 0)->(4162520815, -919640432) [ 3.246538] registered taskstats version 1 [ 3.248356] Loading compiled-in X.509 certificates [ 3.250759] zswap: loaded using pool lzo/zbud [ 3.277380] Key type big_key registered [ 3.290642] Key type encrypted registered [ 3.291782] ima: No TPM chip found, activating TPM-bypass! [ 3.293628] ima: Allocated hash algorithm: sha1 [ 3.295190] ima: No architecture policies found [ 3.296590] evm: Initialising EVM extended attributes: [ 3.298270] evm: security.selinux [ 3.299286] evm: security.ima [ 3.300389] evm: security.capability [ 3.301511] evm: HMAC attrs: 0x1 [ 3.303858] rtc_cmos 00:05: setting system clock to 2026-08-31 08:19:58 UTC (1788164398) [ 3.311350] debug: unmapping init [mem 0xffffffffbac03000-0xffffffffbadfffff] [ 3.314039] debug: unmapping init [mem 0xffffffffb9982000-0xffffffffb9c58fff] [ 3.320301] Write protecting the kernel read-only data: 28672k [ 3.323961] debug: unmapping init [mem 0xffffffffb8003000-0xffffffffb81fffff] [ 3.326785] debug: unmapping init [mem 0xffffffffb8914000-0xffffffffb89fffff] [ 3.364432] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.372435] systemd[1]: Detected virtualization kvm. [ 3.374411] systemd[1]: Detected architecture x86-64. [ 3.376478] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.405725] systemd[1]: No hostname configured. [ 3.407565] systemd[1]: Set hostname to . [ 3.410622] random: systemd: uninitialized urandom read (16 bytes read) [ 3.412611] systemd[1]: Initializing machine ID from random generator. [ 3.524923] random: systemd: uninitialized urandom read (16 bytes read) [ 3.526922] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.530060] random: systemd: uninitialized urandom read (16 bytes read) [ 3.531601] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.534480] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.073815] device-mapper: uevent: version 1.0.3 [ 4.076741] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.820273] random: fast init done [ 4.820735] virtio_net virtio0 ens2: renamed from eth0 [ 4.924156] scsi host0: ata_piix [ 4.964270] scsi host1: ata_piix [ 4.965891] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.968112] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.587372] dracut-initqueue[578]: RTNETLINK answers: File exists [ 9.808220] random: crng init done [ 9.809169] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.207684] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.324377] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.571150] SELinux: Disabled at runtime. [ 11.641712] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.650619] systemd[1]: Detected virtualization kvm. [ 11.652855] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.114909] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.118460] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.122614] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.126621] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.131259] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.136906] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.141930] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target RPC Port Mapper. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Switch Root. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Starting Create list of[ 12.325189] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS required st…ce nodes for the current kernel... Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.665883] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.039443] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.176301] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.190699] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.223423] EDAC sbridge: Ver: 1.1.2 [ 14.938677] Key type dns_resolver registered [ 15.262051] NFS: Registering the id_resolver key type [ 15.264210] Key type id_resolver registered [ 15.265953] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg214-server login: [ 41.443795] libcfs: loading out-of-tree module taints kernel. [ 41.469757] Key type ._llcrypt registered [ 41.471317] Key type .llcrypt registered [ 41.538804] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 55.037358] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 57.071914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 57.082769] alg: No test for adler32 (adler32-zlib) [ 58.442871] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 59.151405] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 60.879191] Key type lgssc registered [ 62.746663] Lustre: Echo OBD driver; http://www.lustre.org/ [ 81.039915] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 104.422078] hrtimer: interrupt took 4326672 ns [ 124.234600] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 140.799661] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 140.906481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 142.352700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 142.391585] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 142.608176] Lustre: lustre-MDT0000: new disk, initializing [ 142.739566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 142.754660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 148.478861] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 165.506308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.660183] Lustre: 6491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 165.702814] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 165.708047] Lustre: Skipped 1 previous similar message [ 165.875156] Lustre: lustre-MDT0001: new disk, initializing [ 165.993274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 166.054232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 166.088509] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 171.506367] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 177.421560] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 188.991218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.180310] Lustre: lustre-OST0000: new disk, initializing [ 189.184095] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 189.197947] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 189.202374] Lustre: 8428:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 189.238244] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 194.612905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 194.629929] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 194.680068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 198.677600] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 215.318889] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 215.504756] Lustre: lustre-OST0001: new disk, initializing [ 215.507462] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 215.511058] Lustre: 9502:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 215.581568] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 222.311336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 222.326593] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 222.443492] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 224.302458] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.917253] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 244.482389] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 251.877453] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing check_logdir /tmp/testlogs/ [ 259.066417] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing yml_node [ 264.200714] Lustre: DEBUG MARKER: Client: 2.17.56.54 [ 266.367253] Lustre: DEBUG MARKER: MDS: 2.17.56.54 [ 268.712572] Lustre: DEBUG MARKER: OSS: 2.17.56.54 [ 270.054465] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 31 04:24:24 EDT 2026 [ 287.663373] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 290.010468] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 309.216440] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 309.230422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 309.253950] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 312.542931] Lustre: server umount lustre-MDT0000 complete [ 317.520523] LustreError: 7447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788164712 with bad export cookie 10857856037325541982 [ 317.526573] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 317.539065] LustreError: 7447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 319.971702] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.973296] LustreError: 6498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 320.019154] Lustre: Skipped 3 previous similar messages [ 320.022048] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 320.061848] LustreError: 6498:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 320.082388] Lustre: Skipped 4 previous similar messages [ 322.976136] LustreError: 6497:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 322.979854] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 323.032760] LustreError: 6497:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 323.084116] Lustre: Skipped 1 previous similar message [ 324.485291] Lustre: server umount lustre-MDT0001 complete [ 329.424221] Lustre: server umount lustre-OST0000 complete [ 333.951395] Lustre: server umount lustre-OST0001 complete [ 341.306092] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 353.269757] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 392.173058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 392.374375] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 392.403939] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 392.456993] Lustre: lustre-MDT0000: new disk, initializing [ 392.523722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 392.539778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 397.502937] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 408.580678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 408.740714] Lustre: 15056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 408.823732] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 408.831250] Lustre: Skipped 1 previous similar message [ 408.911940] Lustre: lustre-MDT0001: new disk, initializing [ 409.024886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 409.067190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 409.081739] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 413.295825] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 424.186112] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 430.644942] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 438.174993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 438.431970] Lustre: lustre-OST0000: new disk, initializing [ 438.435447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 438.443941] Lustre: 17027:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 438.557922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.936001] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 439.964416] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 440.099293] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 445.039612] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 456.444802] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 462.786490] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 463.059889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 468.357795] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 468.733834] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 476.136357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 476.141894] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 476.159867] Lustre: Skipped 1 previous similar message [ 476.168784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 476.170956] LustreError: Skipped 1 previous similar message [ 476.191963] Lustre: Skipped 1 previous similar message [ 477.934542] Lustre: server umount lustre-OST0000 complete [ 485.857138] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 485.865136] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.887921] Lustre: Skipped 1 previous similar message [ 485.897922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 487.967789] Lustre: server umount lustre-MDT0000 complete [ 490.984068] LustreError: 15050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788164886 with bad export cookie 10857856037325543515 [ 490.997383] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 491.332889] Lustre: server umount lustre-MDT0001 complete [ 496.247851] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:28:09 (1788164889) [ 504.121661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 504.600547] LustreError: 19292:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 504.675132] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 509.334586] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 509.921139] LustreError: 19293:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.017504] LustreError: 19292:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.018860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 521.396773] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 529.794444] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 534.985493] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 542.062363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.193772] LustreError: 21113:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 548.733753] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 557.471048] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 561.208173] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 568.800298] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 568.808305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 568.832774] Lustre: Skipped 1 previous similar message [ 568.837877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 568.859022] Lustre: Skipped 2 previous similar messages [ 573.570651] Lustre: server umount lustre-MDT0000 complete [ 574.437129] LustreError: 19288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 574.459645] LustreError: 19288:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 578.976527] LustreError: 19274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788164974 with bad export cookie 10857856037325544397 [ 578.999578] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 579.003097] LustreError: 19274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 579.569662] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 579.581231] Lustre: Skipped 2 previous similar messages [ 579.863689] Lustre: server umount lustre-MDT0001 complete [ 587.033792] Lustre: server umount lustre-OST0000 complete [ 606.666766] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 616.657735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 617.051754] LustreError: 23671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 617.067778] LustreError: 23671:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 617.105518] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 617.111360] Lustre: Skipped 2 previous similar messages [ 621.678922] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 630.268788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 635.312519] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 638.005810] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 644.584296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 651.098355] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 660.382807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 660.730528] Lustre: lustre-OST0001: new disk, initializing [ 660.742906] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 660.753590] Lustre: 25986:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 661.847054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 661.879354] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 662.030510] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 667.502830] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 677.112582] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 680.551476] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 692.192710] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 692.202857] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 692.215657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 692.225780] Lustre: Skipped 3 previous similar messages [ 697.932199] Lustre: server umount lustre-MDT0000 complete [ 698.855201] LustreError: 24264:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 698.884814] LustreError: 24264:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 701.429467] LustreError: 23654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165096 with bad export cookie 10857856037325545244 [ 701.440070] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 701.444783] LustreError: 23654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 701.887752] Lustre: server umount lustre-MDT0001 complete [ 715.655604] Lustre: server umount lustre-OST0000 complete [ 729.332469] Lustre: server umount lustre-OST0001 complete [ 748.930820] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 762.041505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 762.722277] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 762.740730] Lustre: Skipped 3 previous similar messages [ 768.829652] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 778.707187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 783.497464] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 786.815586] Lustre: 30157:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 793.700612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 800.605891] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 810.703617] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 819.772760] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 828.032847] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 832.321562] Lustre: 32038:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 840.673565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 840.683053] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 840.694734] Lustre: Skipped 3 previous similar messages [ 840.704539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 840.707314] Lustre: Skipped 4 previous similar messages [ 846.557464] Lustre: server umount lustre-MDT0000 complete [ 846.821846] LustreError: 29009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 846.846604] LustreError: 29009:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 851.391324] LustreError: 32040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165246 with bad export cookie 10857856037325546539 [ 851.396298] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 851.399292] LustreError: 32040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 851.923115] Lustre: server umount lustre-MDT0001 complete [ 867.682381] Lustre: server umount lustre-OST0000 complete [ 868.319727] Lustre: 3623:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788165247/real 1788165247] req@ffff90c1bb2e0380 x1875026330817408/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788165263 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 872.767770] Lustre: server umount lustre-OST0001 complete [ 883.481269] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:34:37 (1788165277) [ 918.680707] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 934.033113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 940.224875] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 949.359000] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 954.519921] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 957.879686] Lustre: 36010:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 965.416048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 971.599504] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 980.400292] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 987.996834] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 995.870935] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1000.336319] Lustre: 37890:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1006.565441] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1006.572295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1006.596147] Lustre: Skipped 5 previous similar messages [ 1006.598422] Lustre: Skipped 4 previous similar messages [ 1011.172317] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1012.568363] Lustre: server umount lustre-MDT0000 complete [ 1016.087020] LustreError: 34842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165411 with bad export cookie 10857856037325547645 [ 1016.090930] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1016.096672] LustreError: 34842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1016.723368] Lustre: server umount lustre-MDT0001 complete [ 1030.838563] Lustre: server umount lustre-OST0000 complete [ 1032.159517] Lustre: 3624:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788165411/real 1788165411] req@ffff90c0858c2a00 x1875026330866304/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788165427 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1035.230688] Lustre: server umount lustre-OST0001 complete [ 1044.212151] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:37:18 (1788165438) [ 1060.951390] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1073.113871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1073.489828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1073.497480] Lustre: Skipped 7 previous similar messages [ 1078.903398] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1087.526400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1092.381378] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1095.277531] Lustre: 41182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1102.993859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1109.400173] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1112.758844] LustreError: 41536:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1112.781799] LustreError: 41536:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 1118.440320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1124.625994] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1132.058937] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1135.764509] Lustre: 43059:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1136.703476] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1136.708338] Lustre: Skipped 1 previous similar message [ 1144.289150] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1144.323887] Lustre: Skipped 6 previous similar messages [ 1145.847418] Lustre: server umount lustre-MDT0000 complete [ 1149.704563] LustreError: 40012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165544 with bad export cookie 10857856037325548751 [ 1149.713478] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1149.720785] LustreError: 40012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1156.459482] Lustre: server umount lustre-MDT0001 complete [ 1166.443031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1179.821825] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1184.763109] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1188.285036] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1193.360162] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1193.854556] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1193.870789] Lustre: Skipped 1 previous similar message [ 1201.008433] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1202.698528] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1208.211734] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1209.952731] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1214.433328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1220.916292] Lustre: server umount lustre-MDT0000 complete [ 1224.104521] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165619 with bad export cookie 10857856037325549843 [ 1224.108611] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1224.122146] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1224.483859] Lustre: server umount lustre-MDT0001 complete [ 1238.640051] Lustre: server umount lustre-OST0000 complete [ 1240.032073] Lustre: 3625:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788165619/real 1788165619] req@ffff90c188954700 x1875026330947072/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788165635 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1242.756549] Lustre: server umount lustre-OST0001 complete [ 1252.146358] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:40:46 (1788165646) [ 1266.965423] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1277.278160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1282.869226] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1294.082065] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1298.783207] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1301.844948] Lustre: 48818:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1307.843992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1315.641227] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1324.496693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1332.785836] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1341.285634] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1346.523589] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1360.890806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1360.910600] Lustre: Skipped 25 previous similar messages [ 1366.016909] Lustre: server umount lustre-MDT0000 complete [ 1370.560264] LustreError: 47646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788165765 with bad export cookie 10857856037325550669 [ 1370.562606] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1370.577740] LustreError: 47646:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1371.049634] Lustre: server umount lustre-MDT0001 complete [ 1385.951231] Lustre: server umount lustre-OST0000 complete [ 1387.296163] Lustre: 3625:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788165766/real 1788165766] req@ffff90c1bb9d2d80 x1875026331005312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788165782 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1389.747197] Lustre: server umount lustre-OST0001 complete [ 1400.062631] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:43:13 (1788165793) [ 1402.976076] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1405.076497] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:43:18 (1788165798) [ 1416.392166] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 1425.989455] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1449.863970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1450.348072] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1450.488190] Lustre: lustre-MDT0000: new disk, initializing [ 1450.663023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1455.624809] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1465.538518] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1471.757748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1471.972097] Lustre: lustre-OST0000: new disk, initializing [ 1471.974267] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1471.979189] Lustre: Skipped 1 previous similar message [ 1471.982575] Lustre: 54899:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1474.025662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1474.046281] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1474.083174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1478.774382] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1488.945757] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1491.788987] Lustre: DEBUG MARKER: create test files [ 1530.344290] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1530.355847] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1530.386729] Lustre: Skipped 12 previous similar messages [ 1534.073427] Lustre: server umount lustre-OST0000 complete [ 1538.579832] Lustre: server umount lustre-MDT0000 complete [ 1550.545694] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 1553.680425] Key type lgssc unregistered [ 1554.153204] LNet: 56531:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1554.159760] LNetError: 56531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1554.172713] LNet: Removed LNI 192.168.202.114@tcp [ 1555.455265] Key type .llcrypt unregistered [ 1555.460109] Key type ._llcrypt unregistered [ 1585.983939] Key type ._llcrypt registered [ 1585.989487] Key type .llcrypt registered [ 1586.111065] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1588.230586] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1588.265046] alg: No test for adler32 (adler32-zlib) [ 1589.410401] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 1589.779117] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 1591.544255] Key type lgssc registered [ 1593.155079] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1600.909735] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1600.934396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1602.481640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1606.297698] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1613.844437] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1621.462597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1621.751206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1626.890372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1627.071092] Lustre: 59273:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1627.091631] Lustre: 59273:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1627.097026] Lustre: 59273:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1627.110142] Lustre: 59273:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1627.119524] Lustre: 59273:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1627.135658] Lustre: 59273:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1630.115525] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1641.591423] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1648.520857] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1654.968666] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1688.551545] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1688.552647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1688.575723] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1692.834432] Lustre: server umount lustre-OST0000 complete [ 1698.013172] Lustre: server umount lustre-MDT0000 complete [ 1710.530953] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 1714.541918] Key type lgssc unregistered [ 1714.972881] LNet: 60913:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1714.982995] LNetError: 60913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1715.010711] LNet: Removed LNI 192.168.202.114@tcp [ 1716.163089] Key type .llcrypt unregistered [ 1716.170553] Key type ._llcrypt unregistered [ 1749.803972] Key type ._llcrypt registered [ 1749.806634] Key type .llcrypt registered [ 1749.903503] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1750.777769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1750.940023] alg: No test for adler32 (adler32-zlib) [ 1752.183464] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 1752.465317] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 1754.151651] Key type lgssc registered [ 1755.459936] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1764.549282] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1764.566057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1765.999193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1772.028307] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1780.469903] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1788.553710] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1788.831749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1790.906328] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1791.082590] Lustre: 63750:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1791.114136] Lustre: 63750:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1791.133185] Lustre: 63750:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1791.145121] Lustre: 63750:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1791.161767] Lustre: 63750:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1791.167388] Lustre: 63750:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1796.150294] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1804.499712] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1810.150395] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1821.665184] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1821.694503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1825.648771] Lustre: server umount lustre-OST0000 complete [ 1830.044432] Lustre: server umount lustre-MDT0000 complete [ 1844.390360] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 1848.312159] Key type lgssc unregistered [ 1848.640767] LNet: 65311:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1848.651678] LNetError: 65311:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1848.667603] LNet: Removed LNI 192.168.202.114@tcp [ 1849.572996] Key type .llcrypt unregistered [ 1849.575207] Key type ._llcrypt unregistered [ 1868.760973] Key type ._llcrypt registered [ 1868.762839] Key type .llcrypt registered [ 1868.858702] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 1885.438241] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 1886.780953] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1886.824442] alg: No test for adler32 (adler32-zlib) [ 1887.969881] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 1888.195449] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 1889.839204] Key type lgssc registered [ 1891.282389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1929.480355] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1929.527169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1930.769317] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1930.786431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1930.846959] Lustre: lustre-MDT0000: new disk, initializing [ 1930.966633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1931.002639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1936.714320] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1948.711956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1948.878760] Lustre: 68466:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1948.957690] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1948.964902] Lustre: Skipped 1 previous similar message [ 1949.044431] Lustre: lustre-MDT0001: new disk, initializing [ 1949.151676] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1949.204168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1949.221106] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1954.601769] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1965.010883] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1970.802893] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1977.801769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1978.097662] Lustre: lustre-OST0000: new disk, initializing [ 1978.101719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1978.107592] Lustre: 70437:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1978.294366] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1985.342887] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 1987.706696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1987.718310] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1987.832206] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1995.031768] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1999.820157] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2000.133954] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2004.746044] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2005.008584] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2008.545706] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2008.564316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2008.605071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2008.611453] Lustre: Skipped 1 previous similar message [ 2013.668687] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2013.683598] Lustre: Skipped 1 previous similar message [ 2018.785641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2021.343168] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2021.594867] Lustre: server umount lustre-OST0000 complete [ 2025.966357] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2025.981171] LustreError: Skipped 1 previous similar message [ 2025.996181] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2026.014946] Lustre: Skipped 1 previous similar message [ 2026.025108] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2026.030495] Lustre: Skipped 1 previous similar message [ 2029.047544] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2034.162698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2034.173894] Lustre: Skipped 3 previous similar messages [ 2040.287363] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2040.571308] Lustre: server umount lustre-MDT0000 complete [ 2045.510546] LustreError: 68458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788166440 with bad export cookie 9497118700978349246 [ 2045.523631] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2045.528588] LustreError: 68458:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2046.202418] Lustre: server umount lustre-MDT0001 complete [ 2056.393990] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:54:09 (1788166449) [ 2069.245381] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 2111.898504] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 2122.051840] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 2158.771484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2158.952979] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2158.985828] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2159.061719] Lustre: lustre-MDT0000: new disk, initializing [ 2159.133106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2159.145402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2163.709293] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2175.438706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.630026] Lustre: 76463:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2175.700502] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2175.704131] Lustre: Skipped 1 previous similar message [ 2175.848497] Lustre: lustre-MDT0001: new disk, initializing [ 2175.970721] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2176.028215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2176.046949] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2180.433740] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2190.572774] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2195.988667] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2203.132341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2203.522751] Lustre: lustre-OST0000: new disk, initializing [ 2203.541860] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2203.559724] Lustre: 78432:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2203.722607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2211.522821] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2212.918942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2212.923684] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2213.023213] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2223.206614] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2228.510729] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2228.781377] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2233.552712] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2233.839210] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2238.434451] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2238.438110] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2238.454734] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2238.463966] Lustre: Skipped 2 previous similar messages [ 2250.207084] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2250.600259] Lustre: server umount lustre-OST0000 complete [ 2257.890432] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2257.893924] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2257.906099] Lustre: Skipped 1 previous similar message [ 2257.914376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2257.926351] Lustre: Skipped 5 previous similar messages [ 2260.997708] Lustre: server umount lustre-MDT0000 complete [ 2264.544097] LustreError: 76474:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2264.563309] LustreError: 76474:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2265.003411] LustreError: 77258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788166660 with bad export cookie 9497118700978350128 [ 2265.011464] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2265.384404] Lustre: server umount lustre-MDT0001 complete [ 2273.220590] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:57:47 (1788166667) [ 2280.609879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2281.377343] LustreError: 80848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2281.506211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2286.565249] LustreError: 80849:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.676797] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2291.687229] LustreError: 80848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2295.845890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2300.959909] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2308.619489] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2314.031836] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2320.494789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2320.931031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2320.940768] Lustre: Skipped 1 previous similar message [ 2327.262322] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2333.940290] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2337.947794] Lustre: 83519:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 143040a3-f24b-469b-98a9-f652a88eca29 at adminstrative request [ 2342.398118] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.418643] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2352.625702] Lustre: MGS: Client 143040a3-f24b-469b-98a9-f652a88eca29 (at 0@lo) reconnecting [ 2352.639645] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x83cc8b240188aba2 to 0x83cc8b240188ae42 [ 2352.652562] Lustre: MGC192.168.202.114@tcp: Connection restored to 0@lo (at 0@lo) [ 2374.469353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2374.717601] Lustre: lustre-OST0001: new disk, initializing [ 2374.721166] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2374.737299] Lustre: 83912:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2380.396146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2380.410026] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2380.504996] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2381.096384] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2390.653409] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2396.128245] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2396.137354] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2396.140228] LustreError: Skipped 1 previous similar message [ 2396.141639] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2396.141655] Lustre: Skipped 2 previous similar messages [ 2396.165784] Lustre: Skipped 2 previous similar messages [ 2407.391643] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2407.507797] Lustre: server umount lustre-OST0001 complete [ 2411.488761] LustreError: 83003:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.514829] LustreError: 83003:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2413.024818] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2413.030835] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2421.730272] LustreError: 82668:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2421.747081] LustreError: 82668:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2426.335173] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2426.443478] Lustre: server umount lustre-OST0000 complete [ 2435.891454] Lustre: server umount lustre-MDT0000 complete [ 2439.110512] LustreError: 80830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788166834 with bad export cookie 9497118700978351682 [ 2439.112851] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2439.120232] LustreError: 80830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2439.521936] Lustre: server umount lustre-MDT0001 complete [ 2446.639500] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:00:40 (1788166840) [ 2447.839454] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2450.059353] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:00:43 (1788166843) [ 2451.721637] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2453.630579] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:00:47 (1788166847) [ 2455.047518] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2456.740245] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:00:50 (1788166850) [ 2458.259647] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2460.103452] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:00:54 (1788166854) [ 2461.473746] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2467.181409] loop: module loaded [ 2467.195743] loop0: detected capacity change from 0 to 4294967296 [ 2467.284862] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2469.106778] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2472.859649] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2477.210063] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2504.529672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2504.578785] LustreError: 88051:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2504.587046] LustreError: 88051:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2504.596115] LustreError: 88051:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2504.602656] LustreError: 88051:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2523.230185] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 2531.487787] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 2566.422980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2566.801121] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2566.840457] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2567.001051] Lustre: lustre-MDT0000: new disk, initializing [ 2567.140540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2567.146492] Lustre: Skipped 1 previous similar message [ 2567.160522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2572.068872] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2584.654598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2584.872368] Lustre: 90916:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2585.497429] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2590.626210] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2599.965582] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2605.685343] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2611.864568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2612.226525] Lustre: 92885:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2617.599728] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2617.962879] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2618.144648] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2627.367985] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2633.537990] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2633.789773] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2640.009400] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2640.198733] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2643.950536] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2643.954086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2643.960867] LustreError: Skipped 2 previous similar messages [ 2643.963848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2643.981474] Lustre: Skipped 4 previous similar messages [ 2644.000384] Lustre: Skipped 14 previous similar messages [ 2648.411740] Lustre: server umount lustre-OST0000 complete [ 2659.245269] Lustre: server umount lustre-MDT0000 complete [ 2659.298994] LustreError: 90928:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2659.321966] LustreError: 90928:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2662.835289] LustreError: 90910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788167058 with bad export cookie 9497118700978352298 [ 2662.838716] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2662.845609] LustreError: 90910:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2663.192782] Lustre: server umount lustre-MDT0001 complete [ 2670.682250] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:04:24 (1788167064) [ 2677.809363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2683.058466] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2690.680165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2695.397907] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2702.782936] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2707.620687] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2714.191804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2714.619151] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2714.624984] Lustre: Skipped 4 previous similar messages [ 2721.819309] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2729.663842] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2734.814136] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2735.050485] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2740.020314] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2740.360675] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2747.419245] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2747.572612] Lustre: lustre-OST0001: new disk, initializing [ 2747.575185] Lustre: Skipped 2 previous similar messages [ 2747.577606] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2747.580531] Lustre: Skipped 3 previous similar messages [ 2747.584518] Lustre: 98666:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2754.768185] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2755.231396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2755.240886] Lustre: Skipped 2 previous similar messages [ 2755.250836] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2755.360830] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2764.092409] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2769.082971] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2769.272492] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2773.980300] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2774.314269] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2793.471373] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2813.916179] Lustre: Failing over lustre-MDT0000 [ 2813.926517] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -19 [ 2813.941061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2813.950874] Lustre: Skipped 2 previous similar messages [ 2813.966157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2813.973987] Lustre: Skipped 2 previous similar messages [ 2814.407395] Lustre: server umount lustre-MDT0000 complete [ 2816.997561] LustreError: 95297:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2817.015574] LustreError: 95297:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 2833.376652] Lustre: 66688:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788167212/real 1788167212] req@000000006a198a3e x1875028248641536/t0(0) o400->MGC192.168.202.114@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788167228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2833.445875] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2840.702754] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2840.704634] LDISKFS-fs (dm-0): recovery complete [ 2840.722809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2842.847148] LustreError: 101482:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2842.860293] LustreError: 101482:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000d2c8f8b1 x1875028248647168/t0(0) o250->MGC192.168.202.114@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1788167238 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2842.880258] LustreError: 101482:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2844.653556] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x83cc8b24018978bd [ 2844.668374] Lustre: MGC192.168.202.114@tcp: Connection restored to 0@lo (at 0@lo) [ 2844.930589] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2849.197220] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2849.206269] Lustre: 95292:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2849.580901] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 2850.275083] Lustre: 95297:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2850.297849] Lustre: 95297:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2850.314807] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2850.415193] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2850.747111] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.095393] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2851.109536] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2851.431136] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.441960] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2851.455223] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2851.474559] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2852.128090] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2852.157564] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2852.487126] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2852.499121] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2853.490089] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2853.500652] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 2854.160513] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2854.180938] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2854.495172] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2854.502260] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2857.681770] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2857.687408] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2858.324890] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2858.351489] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2858.671146] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2858.680112] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2865.687887] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2865.693649] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2866.330329] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2866.354303] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2866.696047] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2866.708045] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2881.688886] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2881.699695] Lustre: 101518:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 2882.353706] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.361858] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2883.000951] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2883.018310] LustreError: 101518:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2909.023569] Lustre: 101518:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2909.035116] Lustre: 101518:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2909.046207] Lustre: 101518:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 288072fd-b4f2-4033-9da7-8b3a5f9decf0@192.168.202.14@tcp [ 2909.063807] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2909.070305] Lustre: 101518:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000476ff5c4 x1875028231656576/t0(8589934780) o35->288072fd-b4f2-4033-9da7-8b3a5f9decf0@192.168.202.14@tcp:399/0 lens 392/0 e 0 to 0 dl 1788167314 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2909.117408] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2909.131674] Lustre: lustre-MDT0000: Denying connection for new client 288072fd-b4f2-4033-9da7-8b3a5f9decf0 (at 192.168.202.14@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2909.161247] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2909.186495] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2909.192487] Lustre: Skipped 2 previous similar messages [ 2909.193416] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2909.227687] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2909.233295] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2925.025646] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2928.658427] Lustre: server umount lustre-OST0000 complete [ 2932.196092] LustreError: 99050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2932.206879] LustreError: 99050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 2938.760824] Lustre: server umount lustre-OST0001 complete [ 2945.005755] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2945.032291] Lustre: Skipped 7 previous similar messages [ 2948.996698] Lustre: server umount lustre-MDT0000 complete [ 2952.791418] LustreError: 95279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788167347 with bad export cookie 9497118700978403517 [ 2952.796610] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2952.800028] LustreError: 95279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2953.189710] Lustre: server umount lustre-MDT0001 complete [ 2961.532374] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:09:15 (1788167355) [ 2968.442985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2968.448212] Lustre: *** cfs_fail_loc=197, val=0*** [ 2983.967906] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2989.087611] LustreError: 103492:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.114@tcp: failed processing log, type 4: rc = -110 [ 2994.255452] Lustre: *** cfs_fail_loc=197, val=0*** [ 2994.261711] Lustre: Skipped 65 previous similar messages [ 3014.815490] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3014.835028] Lustre: Skipped 2 previous similar messages [ 3021.482560] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3028.895412] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3031.607432] Lustre: server umount lustre-OST0000 complete [ 3039.379891] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:10:33 (1788167433) [ 3060.398041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3061.036068] LustreError: 105112:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3061.057070] LustreError: 105112:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3065.053448] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3075.078472] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3078.813993] LustreError: 105143:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3078.822801] LustreError: 105143:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3078.830798] LustreError: 105143:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3079.094952] Lustre: server umount lustre-MDT0000 complete [ 3108.748372] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 3117.789037] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 3152.125589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3152.354696] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3152.377306] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3152.465300] Lustre: lustre-MDT0000: new disk, initializing [ 3152.544928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3156.661063] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3168.222597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3168.353688] Lustre: 109314:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3168.572341] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3172.825121] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3181.305609] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3186.637600] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3192.168931] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3192.387338] Lustre: 111284:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3197.891491] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3201.709910] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3205.207864] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3209.199203] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3209.365170] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3213.589352] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3213.766337] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3217.386457] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3217.408677] LustreError: Skipped 4 previous similar messages [ 3217.411115] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3217.420445] Lustre: Skipped 1 previous similar message [ 3217.428129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3217.436636] Lustre: Skipped 11 previous similar messages [ 3221.617541] Lustre: server umount lustre-OST0000 complete [ 3231.960392] Lustre: server umount lustre-MDT0000 complete [ 3235.302523] LustreError: 109308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788167630 with bad export cookie 9497118700978414213 [ 3235.303211] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3235.309279] LustreError: 109308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3235.707774] Lustre: server umount lustre-MDT0001 complete [ 3243.958708] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:13:58 (1788167638) [ 3251.773498] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 3254.234895] Key type lgssc unregistered [ 3254.492236] LNet: 113782:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3254.500041] LNetError: 113782:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3254.512452] LNet: Removed LNI 192.168.202.114@tcp [ 3255.403150] Key type .llcrypt unregistered [ 3255.406714] Key type ._llcrypt unregistered [ 3273.382984] Key type ._llcrypt registered [ 3273.385532] Key type .llcrypt registered [ 3273.461452] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 3290.457290] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 3292.834333] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3292.978408] alg: No test for adler32 (adler32-zlib) [ 3294.222927] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 3294.551103] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 3296.247465] Key type lgssc registered [ 3297.366854] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3333.739057] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3333.767997] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3334.999328] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3335.021777] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3335.142283] Lustre: lustre-MDT0000: new disk, initializing [ 3335.237551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3335.253283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3339.322722] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3349.588085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3349.797840] Lustre: lustre-OST0000: new disk, initializing [ 3349.801616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3349.807604] Lustre: Skipped 1 previous similar message [ 3349.815047] Lustre: 117958:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3349.896085] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3350.952967] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3350.961725] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3350.991203] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3355.918705] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3366.555479] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3366.849454] Lustre: lustre-OST0001: new disk, initializing [ 3366.858348] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3366.870464] Lustre: 118820:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3366.978873] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3373.017369] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3373.756491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3373.768751] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3373.824633] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3384.314146] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3384.321825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3384.349939] Lustre: Skipped 1 previous similar message [ 3384.371499] Lustre: Skipped 1 previous similar message [ 3388.727929] Lustre: server umount lustre-MDT0000 complete [ 3394.869457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3398.882840] LustreError: 117957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788167794 with bad export cookie 5944646398069068270 [ 3398.887434] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3398.910770] LustreError: 117957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3398.979312] Lustre: server umount lustre-OST0000 complete [ 3402.148498] Lustre: server umount lustre-OST0001 complete [ 3410.303775] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:16:43 (1788167803) [ 3430.164779] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:17:04 (1788167824) [ 3438.265901] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 3459.229910] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:17:33 (1788167853) [ 3492.245693] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 3500.692165] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 3531.781490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3532.013140] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3532.035569] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3532.106909] Lustre: lustre-MDT0000: new disk, initializing [ 3532.180347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3532.200369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3536.642081] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3546.816046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3546.956213] Lustre: 126570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3547.013683] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3547.017452] Lustre: Skipped 1 previous similar message [ 3547.111744] Lustre: lustre-MDT0001: new disk, initializing [ 3547.223762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3547.258180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3547.270164] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3551.235338] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3559.142502] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3564.207392] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3569.983908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3570.210534] Lustre: lustre-OST0000: new disk, initializing [ 3570.214112] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3570.218842] Lustre: 128538:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3570.308901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3575.284440] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3579.928047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3579.935563] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3580.049880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3582.450915] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3586.555446] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3586.801283] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3592.018823] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3592.226400] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3595.232356] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3595.243069] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3595.257836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3600.868686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3600.876790] Lustre: Skipped 1 previous similar message [ 3605.987979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3605.993596] Lustre: Skipped 1 previous similar message [ 3608.036936] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3608.255822] Lustre: server umount lustre-OST0000 complete [ 3613.664782] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3613.675192] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3613.682679] Lustre: Skipped 1 previous similar message [ 3613.690698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3613.699118] Lustre: Skipped 1 previous similar message [ 3616.234393] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3617.534253] Lustre: server umount lustre-MDT0000 complete [ 3620.581295] LustreError: 130081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168015 with bad export cookie 5944646398069069495 [ 3620.590177] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3620.956485] Lustre: server umount lustre-MDT0001 complete [ 3627.751513] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:20:22 (1788168022) [ 3633.541531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3634.003174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3638.749461] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3639.265665] LustreError: 130952:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3639.282699] LustreError: 130952:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3644.387676] LustreError: 130951:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3647.069896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3652.365413] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3661.369452] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3667.619580] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3676.257292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3676.647417] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3676.656493] Lustre: Skipped 1 previous similar message [ 3684.170105] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3691.502743] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3698.969956] Lustre: *** cfs_fail_loc=159, val=0*** [ 3715.544863] Lustre: lustre-MDT0000: Client 7c0ee5b8-51c8-4e1a-95d8-26f3605d5650 (at 192.168.202.14@tcp) reconnecting [ 3716.995158] Lustre: *** cfs_fail_loc=159, val=0*** [ 3716.996738] Lustre: Skipped 5 previous similar messages [ 3733.477989] Lustre: lustre-MDT0000: Client 7c0ee5b8-51c8-4e1a-95d8-26f3605d5650 (at 192.168.202.14@tcp) reconnecting [ 3740.341565] Lustre: server umount lustre-OST0000 complete [ 3744.739835] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3744.753256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3744.771244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3744.782657] Lustre: Skipped 2 previous similar messages [ 3753.509768] Lustre: server umount lustre-MDT0000 complete [ 3756.830043] LustreError: 130932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168152 with bad export cookie 5944646398069070377 [ 3756.836430] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3756.841407] LustreError: 130932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3767.463293] Lustre: server umount lustre-MDT0001 complete [ 3779.927985] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 3783.384061] Key type lgssc unregistered [ 3783.661708] LNet: 134733:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3783.669513] LNetError: 134733:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3784.743691] LNet: Removed LNI 192.168.202.114@tcp [ 3785.387099] Key type .llcrypt unregistered [ 3785.396305] Key type ._llcrypt unregistered [ 3806.370637] Key type ._llcrypt registered [ 3806.375866] Key type .llcrypt registered [ 3806.477750] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 3807.178881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3807.199710] alg: No test for adler32 (adler32-zlib) [ 3808.148896] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 3808.325979] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 3809.951200] Key type lgssc registered [ 3810.899950] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3817.157517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3817.169690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3818.500206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3822.893074] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3828.705104] LustreError: 136126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3828.724520] LustreError: 136126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3830.651642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3831.066970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3836.138039] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3844.632734] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3850.662723] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3857.510333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3857.966870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3863.085767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3864.926534] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 3872.890437] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3885.300799] Lustre: *** cfs_fail_loc=159, val=0*** [ 3900.872450] Lustre: lustre-MDT0000: Client 0fe3db5d-0783-4c98-9c58-3f6da92c7c01 (at 192.168.202.14@tcp) reconnecting [ 3903.215883] Lustre: *** cfs_fail_loc=159, val=0*** [ 3918.803414] Lustre: lustre-MDT0001: Client 0fe3db5d-0783-4c98-9c58-3f6da92c7c01 (at 192.168.202.14@tcp) reconnecting [ 3920.492901] Lustre: *** cfs_fail_loc=159, val=0*** [ 3920.499345] Lustre: Skipped 3 previous similar messages [ 3935.780937] Lustre: lustre-MDT0001: Client 0fe3db5d-0783-4c98-9c58-3f6da92c7c01 (at 192.168.202.14@tcp) reconnecting [ 3952.721555] Lustre: 139188:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3955.597669] Lustre: *** cfs_fail_loc=159, val=0*** [ 3955.599891] Lustre: Skipped 4 previous similar messages [ 3972.068166] Lustre: lustre-MDT0000: Client 7b9aa0b1-c74e-4a6b-800f-09e957f1d3fb (at 192.168.202.14@tcp) reconnecting [ 3973.935147] Lustre: *** cfs_fail_loc=159, val=0*** [ 3973.938203] Lustre: Skipped 14 previous similar messages [ 3990.488459] Lustre: lustre-MDT0000: Client 7b9aa0b1-c74e-4a6b-800f-09e957f1d3fb (at 192.168.202.14@tcp) reconnecting [ 3991.941655] Lustre: 139480:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3998.184033] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3998.185666] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3998.193570] LustreError: Skipped 2 previous similar messages [ 3998.219474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3999.203720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3999.207595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4002.878610] Lustre: server umount lustre-OST0000 complete [ 4009.441245] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4009.446443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4009.466612] Lustre: Skipped 1 previous similar message [ 4009.482194] Lustre: Skipped 2 previous similar messages [ 4012.558555] Lustre: server umount lustre-MDT0000 complete [ 4014.561839] LustreError: 137327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4014.584281] LustreError: 137327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4016.356557] LustreError: 136108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168411 with bad export cookie 4236792905646779953 [ 4016.359358] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4016.367916] LustreError: 136108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4016.708048] Lustre: server umount lustre-MDT0001 complete [ 4028.376860] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4031.942628] Key type lgssc unregistered [ 4032.508088] LNet: 140534:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4032.520820] LNetError: 140534:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4032.551991] LNet: Removed LNI 192.168.202.114@tcp [ 4033.520832] Key type .llcrypt unregistered [ 4033.523634] Key type ._llcrypt unregistered [ 4056.765704] Key type ._llcrypt registered [ 4056.767690] Key type .llcrypt registered [ 4056.856472] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4058.240641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4058.289729] alg: No test for adler32 (adler32-zlib) [ 4059.287846] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4059.507634] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4061.207338] Key type lgssc registered [ 4062.472908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4069.317910] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4069.362238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4070.723677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4073.887041] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4080.099296] LustreError: 141952:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4080.123803] LustreError: 141952:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4081.527366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4081.791079] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4086.008280] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4092.509420] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4096.691379] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4102.271935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4102.605555] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4106.737801] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4107.996873] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4114.932251] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4115.054367] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4121.521040] Lustre: 144531:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4124.429241] Lustre: 144580:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4125.613846] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4126.516103] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4130.153226] Lustre: 144727:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4130.937232] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4131.711344] Lustre: 144825:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4137.442532] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4137.455118] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4137.484261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4140.513262] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4140.520539] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4140.539108] Lustre: Skipped 1 previous similar message [ 4142.460945] Lustre: server umount lustre-OST0000 complete [ 4148.192914] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4148.203051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.213945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4150.753323] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4150.756518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4150.783395] Lustre: Skipped 1 previous similar message [ 4152.164502] Lustre: server umount lustre-MDT0000 complete [ 4155.874577] LustreError: 144543:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4155.904919] LustreError: 144543:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4155.995048] LustreError: 141932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168551 with bad export cookie 6699841068776394126 [ 4156.003276] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4156.383462] Lustre: server umount lustre-MDT0001 complete [ 4167.781333] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4170.520869] Key type lgssc unregistered [ 4170.723806] LNet: 145890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4170.728332] LNetError: 145890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4170.744170] LNet: Removed LNI 192.168.202.114@tcp [ 4171.464489] Key type .llcrypt unregistered [ 4171.466768] Key type ._llcrypt unregistered [ 4199.063289] Key type ._llcrypt registered [ 4199.065322] Key type .llcrypt registered [ 4199.161507] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4200.827109] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4200.904153] alg: No test for adler32 (adler32-zlib) [ 4201.993751] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4202.294980] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4204.007153] Key type lgssc registered [ 4205.586570] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4213.836867] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4213.863759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4215.613938] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4220.661695] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4225.001224] LustreError: 147303:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4225.019609] LustreError: 147303:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4229.439077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4229.722814] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4234.174874] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4240.724968] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4245.450831] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4252.401884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4252.724441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4254.800907] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4257.991737] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4259.897876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4265.405451] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4271.826074] Lustre: *** cfs_fail_loc=159, val=0*** [ 4288.488547] Lustre: lustre-MDT0000: Client 09914939-ce1a-4293-9022-a14fa3cc8a14 (at 192.168.202.14@tcp) reconnecting [ 4305.891614] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4305.901345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4305.903416] LustreError: Skipped 1 previous similar message [ 4305.924298] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4305.931067] Lustre: Skipped 1 previous similar message [ 4310.795861] Lustre: server umount lustre-OST0000 complete [ 4316.129096] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4316.138796] Lustre: Skipped 1 previous similar message [ 4316.142419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4316.151572] Lustre: Skipped 1 previous similar message [ 4320.429459] Lustre: server umount lustre-MDT0000 complete [ 4321.249705] LustreError: 147304:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4321.265728] LustreError: 147304:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4324.290896] LustreError: 147283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168719 with bad export cookie 2131101316040371444 [ 4324.301280] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4324.303037] LustreError: 147283:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4324.632492] Lustre: server umount lustre-MDT0001 complete [ 4333.321301] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4335.796351] Key type lgssc unregistered [ 4336.029850] LNet: 150969:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4336.035946] LNetError: 150969:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4337.063064] LNet: Removed LNI 192.168.202.114@tcp [ 4337.990529] Key type .llcrypt unregistered [ 4337.992541] Key type ._llcrypt unregistered [ 4361.025828] Key type ._llcrypt registered [ 4361.028639] Key type .llcrypt registered [ 4361.118838] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4362.090186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4362.128543] alg: No test for adler32 (adler32-zlib) [ 4363.123965] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4363.303989] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4365.007378] Key type lgssc registered [ 4365.952585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4372.522800] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4372.547599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4373.951695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4378.682342] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4384.228518] LustreError: 152384:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4384.261387] LustreError: 152384:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4386.312925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4386.683547] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4391.159423] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4398.729493] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4404.266060] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4411.483599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4411.947272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4418.022814] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4419.118525] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4424.608971] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4431.410251] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4451.165767] Lustre: lustre-MDT0001: evicting f40d2cb6-8dae-4bff-ab01-51697ea1e6de (at 192.168.202.14@tcp) by administrative request [ 4451.172864] Lustre: Skipped 1 previous similar message [ 4480.483818] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4480.493590] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4480.514541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4482.529517] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4482.543568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4482.548594] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4482.567337] Lustre: Skipped 1 previous similar message [ 4485.612258] Lustre: server umount lustre-OST0000 complete [ 4489.184107] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4489.191558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4489.212135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4492.773570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4492.774649] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4495.057493] Lustre: server umount lustre-MDT0000 complete [ 4497.888765] LustreError: 153114:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4497.894406] LustreError: 153114:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4498.043199] LustreError: 152365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788168893 with bad export cookie 11164411825068886202 [ 4498.064067] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4498.351702] Lustre: server umount lustre-MDT0001 complete [ 4509.424804] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4511.886939] Key type lgssc unregistered [ 4512.135079] LNet: 156459:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4512.143208] LNetError: 156459:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4512.158639] LNet: Removed LNI 192.168.202.114@tcp [ 4513.041200] Key type .llcrypt unregistered [ 4513.043959] Key type ._llcrypt unregistered [ 4581.504392] Key type ._llcrypt registered [ 4581.506869] Key type .llcrypt registered [ 4581.590761] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4582.699881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4582.860047] alg: No test for adler32 (adler32-zlib) [ 4583.972613] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4584.184881] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4585.839281] Key type lgssc registered [ 4586.985090] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4595.875666] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4595.909353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4597.697497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4604.347284] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4606.945896] LustreError: 158947:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4606.971194] LustreError: 158947:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4612.065261] LustreError: 158948:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4614.421936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4615.073347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4619.709321] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4627.816274] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4633.290296] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4640.185876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4640.688000] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4641.723293] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4642.756230] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4648.029987] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4656.206816] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4667.361687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4667.370973] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4667.388592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4668.386084] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4668.401547] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4668.422190] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4673.505807] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4673.514155] Lustre: Skipped 2 previous similar messages [ 4678.636465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4678.653402] Lustre: Skipped 1 previous similar message [ 4680.159195] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4680.411728] Lustre: server umount lustre-OST0000 complete [ 4686.816319] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4686.829143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4686.847076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4688.865275] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4690.962345] Lustre: server umount lustre-MDT0000 complete [ 4693.986898] LustreError: 159679:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4694.025034] LustreError: 159679:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4696.157513] LustreError: 159689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788169091 with bad export cookie 4586807788758161154 [ 4696.161626] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4696.178506] LustreError: 159689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4696.782852] Lustre: server umount lustre-MDT0001 complete [ 4708.757617] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4711.448762] Key type lgssc unregistered [ 4711.696658] LNet: 162497:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4711.705388] LNetError: 162497:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4711.732942] LNet: Removed LNI 192.168.202.114@tcp [ 4712.659571] Key type .llcrypt unregistered [ 4712.661532] Key type ._llcrypt unregistered [ 4737.470262] Key type ._llcrypt registered [ 4737.476305] Key type .llcrypt registered [ 4737.631537] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4738.805609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4738.877423] alg: No test for adler32 (adler32-zlib) [ 4739.909435] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4740.109373] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4741.751192] Key type lgssc registered [ 4742.985961] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4756.476468] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:39:10 (1788169150) [ 4774.201329] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 4785.170710] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4821.509312] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4821.567442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4822.865973] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4822.930781] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4823.019480] Lustre: lustre-MDT0000: new disk, initializing [ 4823.119331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4823.134326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4827.076524] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4837.315426] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4837.456063] Lustre: 166763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4837.498830] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4837.502871] Lustre: Skipped 1 previous similar message [ 4837.611963] Lustre: lustre-MDT0001: new disk, initializing [ 4837.741376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4837.779993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4837.804234] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4842.786215] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4852.470615] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4859.350459] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4868.471130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4869.016484] Lustre: lustre-OST0000: new disk, initializing [ 4869.027486] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4869.034869] Lustre: 168730:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4869.125518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4874.922994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4874.933976] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4875.021020] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4876.913428] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4888.286921] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4900.833146] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4900.837563] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4900.849239] LustreError: Skipped 1 previous similar message [ 4900.850957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4900.889956] Lustre: Skipped 1 previous similar message [ 4904.247736] Lustre: server umount lustre-OST0000 complete [ 4909.536849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4909.541445] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4909.550876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4909.555187] Lustre: Skipped 1 previous similar message [ 4911.086857] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4911.088133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4911.113209] Lustre: Skipped 1 previous similar message [ 4915.655450] Lustre: server umount lustre-MDT0000 complete [ 4920.128212] LustreError: 167566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788169315 with bad export cookie 10207211035107309045 [ 4920.150383] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4920.153515] LustreError: 167566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4920.409205] Lustre: server umount lustre-MDT0001 complete [ 4934.646869] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 4938.147273] Key type lgssc unregistered [ 4938.553835] LNet: 170637:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4938.566361] LNetError: 170637:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4938.594444] LNet: Removed LNI 192.168.202.114@tcp [ 4939.450200] Key type .llcrypt unregistered [ 4939.452112] Key type ._llcrypt unregistered [ 4964.375830] Key type ._llcrypt registered [ 4964.379394] Key type .llcrypt registered [ 4964.497782] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 4965.677356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4965.816753] alg: No test for adler32 (adler32-zlib) [ 4966.867533] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 4967.118967] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 4968.800927] Key type lgssc registered [ 4969.783761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4977.118630] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4977.137974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4978.499819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4982.565235] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 4987.873365] LustreError: 172051:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4987.895311] LustreError: 172051:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4991.183761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4991.615990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4996.197116] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 5003.203730] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5009.139642] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5016.934130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5017.307291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5023.209405] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 5023.488224] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5030.187581] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5040.423265] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5040.453897] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5040.487703] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5040.493817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.511877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5041.421281] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5041.432867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5041.444248] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5041.453047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.466490] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5042.633038] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5042.662440] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5042.675771] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5042.692068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.702214] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5043.577701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.337083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.090844] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5045.099668] Lustre: Skipped 2 previous similar messages [ 5045.114912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5045.124686] Lustre: Skipped 2 previous similar messages [ 5045.129936] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5045.135700] Lustre: Skipped 2 previous similar messages [ 5045.140401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.147402] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5045.152511] Lustre: Skipped 2 previous similar messages [ 5045.972852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.894972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.623404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.484372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.219206] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5049.227581] Lustre: Skipped 4 previous similar messages [ 5049.241563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5049.251444] Lustre: Skipped 4 previous similar messages [ 5049.254847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5049.259771] Lustre: Skipped 4 previous similar messages [ 5049.263501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.275351] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5049.279455] Lustre: Skipped 4 previous similar messages [ 5050.198444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.078808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.281307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.379996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.490987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.557840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.501810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.501381] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5057.509744] Lustre: Skipped 7 previous similar messages [ 5057.527290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5057.540368] Lustre: Skipped 7 previous similar messages [ 5057.549580] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5057.562346] Lustre: Skipped 7 previous similar messages [ 5057.572295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.587988] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5057.597235] Lustre: Skipped 7 previous similar messages [ 5058.559052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.603680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.590168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.397162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.544254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.389603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.262855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.893914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.713523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.668313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.575631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.499817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.377284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.402791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.348589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.370319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.291829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.910836] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5073.914503] Lustre: Skipped 17 previous similar messages [ 5073.924561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5073.945315] Lustre: Skipped 17 previous similar messages [ 5073.949293] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5073.954763] Lustre: Skipped 17 previous similar messages [ 5073.960897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.972409] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5073.982939] Lustre: Skipped 17 previous similar messages [ 5074.641256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.289556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.178253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.128595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.365430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.362318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.691867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.953191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.834647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.860386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.006459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.319687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.342849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.748402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.294177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.779823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.522418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.099277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.310693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.255153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.324622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.691512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.848303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.460328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.880239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.768193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.944845] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5105.949293] Lustre: Skipped 26 previous similar messages [ 5105.967296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5105.975567] Lustre: Skipped 26 previous similar messages [ 5105.979658] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5105.984650] Lustre: Skipped 26 previous similar messages [ 5105.988086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.996615] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5106.003191] Lustre: Skipped 26 previous similar messages [ 5107.156775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.242265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.661752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.287796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.548473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.977833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.106928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.026957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.492519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.730380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.848266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.094451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.886195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.158139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.017437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.184285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.220881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.265549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.244949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.108420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.011929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.904974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.708817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.552387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.496455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.366844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.228031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.108273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.098339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.066552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.941979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.870238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.014303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.840403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.743549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.686989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.820813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.659971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.507478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.292264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.104781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.870076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.815753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.439705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.410881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.231274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.168255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.957324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.827903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.559427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.385033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.152342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.083830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.838658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.835140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.823377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.832399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.052256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.187734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.990043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.911698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.871769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.047168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.871986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.978843] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5169.984893] Lustre: Skipped 64 previous similar messages [ 5170.003201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5170.011184] Lustre: Skipped 64 previous similar messages [ 5170.017031] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5170.021439] Lustre: Skipped 64 previous similar messages [ 5170.023965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.034344] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5170.036918] Lustre: Skipped 64 previous similar messages [ 5170.923690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.827448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.502975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.345852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.152359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.981394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.789360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.700573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.403449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.207647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.059203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.921512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.792361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.569332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.441361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.119927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.154700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.147476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.114673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.843242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.916439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.051305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.894339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.814889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.668342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.794167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.688129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.648971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.408143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.212589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.981493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.770527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.573740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.495182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.207354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.084174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.869298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.864892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.703379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.561610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.447206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.669411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.601333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.412634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.290543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.263785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.052790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.880396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.676743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.418744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.177229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.083028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.866455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.700913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.530728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.286402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.162472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.848969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.573640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.298936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.081139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.793772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.492683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.346621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.321105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.986349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.885855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.588076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.408637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.229433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.095611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.932462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.902149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.848928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.751144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.640502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.485170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.430295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.448316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.314323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.158517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.071963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.784733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.585672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.445494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.343201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.108234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.072389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.098239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.224818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.192723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.257454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.243404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.213591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.320417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.236183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.169873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.046455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.021594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.822514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.788526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.613711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.489137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.279874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.263665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.053509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.794727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.622355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.635554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.428336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.291225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.067873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.831657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.590733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.434533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.239502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.052401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.795446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.621809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.429411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.482868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.485605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.485233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.297222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.000822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.008718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.148454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.497062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.737622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.446968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.267642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.983804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.830976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.537395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.455579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.164421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.139732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.874406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.771096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.667776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.389230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.188043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.091210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.813589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.654722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.564635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.431951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.284274] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5298.290373] Lustre: Skipped 147 previous similar messages [ 5298.309503] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5298.318205] Lustre: Skipped 147 previous similar messages [ 5298.322286] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5298.327401] Lustre: Skipped 147 previous similar messages [ 5298.330961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.340099] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5298.345089] Lustre: Skipped 147 previous similar messages [ 5299.214218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.126520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.996061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.764563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.631608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.464729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.420356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.299656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.028227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.749833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.643808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.535190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.483482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.321193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.174391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.061844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.021668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.879687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.867596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.690892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.697155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.846147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.849085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.154350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.131038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.176393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.093332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.920348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.633405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.410933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.087149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.960405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.635371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.438227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.103270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.802466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.676535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.615865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.681180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.616460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.464084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.124931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.031190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.940322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.072729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.959631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.853856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.762832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.557224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.342899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.231589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.950979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.839046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.588044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.310372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.129791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.980859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.790382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.644541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.409589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.245875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.120969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.000467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.711549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.566759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.369206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.291728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.238293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.050288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.953711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.047906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.816679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.693276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.596203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.502435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.332182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.255882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.188708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.038553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.852406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.793213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.551697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.446113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.225654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.024886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.702103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.682496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.572938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.761649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.876086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.832201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.956915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.877454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.833787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.743704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.043658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.951248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.858631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.027770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.457866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.412034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.757186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.736988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.204389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.761186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.769050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.846034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.789099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.905948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.877853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.578250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.776440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.849473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.801861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.930539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.171859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.520115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.141793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.318194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.116908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.308356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.240766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.176428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.097591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.482181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.352353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.153599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.219646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.165182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.070715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.034872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.146036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.153952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.171510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.045988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.202384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.046719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.992621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.879012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.091190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.919304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.863852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.076460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.023042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.353385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.190549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.196128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.040325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.992485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.847882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.931977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.932395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.126488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.991138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.903634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.921412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.899166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.103629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.384073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.680974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.841660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.926461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.111260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.682384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.591615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.025940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.301968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.753766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.795915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.711237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.446484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.479358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.404654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.412456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.317564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.052411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.804816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.674873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.459411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.295914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.232183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.050379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.912316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.714848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.458927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.489165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.405569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.315306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.213753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.354680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.074210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.121910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.057425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.537087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.995391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.212597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.198761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.126276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.887992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.659595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.493929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.351584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.351881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.454456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.319856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.163138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.386632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.391453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.262074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.169994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.119221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.108679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.122897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.286257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.333266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.541055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.858724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.050957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.099729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.481176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.693779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.716167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.742791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.824208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.959856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.068350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.098727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.930368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.757416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.645299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.675274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.500214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.883051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.150860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.070453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.029658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.997293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.982665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.933548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.883341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.842775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.004421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.873485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.898693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.057302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.005474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.140592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.023405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.075415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.230225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.211046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.308818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.292375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.511815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.505204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.438241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.290229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.359357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.222171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.170110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.055476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.196137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.145726] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5555.150260] Lustre: Skipped 262 previous similar messages [ 5555.159487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5555.172924] Lustre: Skipped 262 previous similar messages [ 5555.176601] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5555.182284] Lustre: Skipped 262 previous similar messages [ 5555.185901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.201715] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5555.206965] Lustre: Skipped 262 previous similar messages [ 5556.025406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.935436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.925260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.657709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.555562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.338646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.281055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.191808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.138238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.119374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.290182] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5565.316343] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5565.327224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.314873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.275238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.083742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.995523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.693828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.612957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.477157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.432623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.380897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.245898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.012184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.880365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.843833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.719231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.649593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.582474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.672077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.612913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.607452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.552632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.734545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.791577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.820843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.714660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.842528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.825475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.688926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.569569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.510964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.490895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.567503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.283444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.107208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.866530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.857605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.711853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.823500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.835275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.848513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.022503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.332877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.292175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.299366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.258186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.080536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.838791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.711914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.558254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.401268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.386076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.243297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.198160] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 5614.210964] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 5614.214135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.085769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.963517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.862236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.955912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.791801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.854796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.650577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.678451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.528633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.655191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.467958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.406529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.205162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.157940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.286364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.436367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.879909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.891788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.795337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.738783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.685435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.576959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.708533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.720333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.562166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.465319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.500624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.426258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.393557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.270593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.254759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.189842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.240430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.977211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.825163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.763778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.753901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.580425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.574831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.579566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.545617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.395578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.324641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.085590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.956820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.730892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.438354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.247262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.164336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.027149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.070355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.854454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.765422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.546390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.348339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.272706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.141786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.029268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.881754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.678207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.606444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.431937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.341868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.298171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.138844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.058191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.973477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.788825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.626112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.456541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.268787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.154825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.982109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.800688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.696901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.517127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.403564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.272500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.132087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.025525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.819881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.489849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.496594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.361198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.236347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.129941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.881176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.659602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.392050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.316460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.146122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.089767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.383881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.320723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.131408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.043354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.789905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.816657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.640127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.390269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.041203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.871571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.806895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.694677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.655184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.542925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.463589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.589294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.369730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.377566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.180309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.163185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.120059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.986567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.142574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.352155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.463234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.546086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.661385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.931902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.066162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.098571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.270403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.471550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.413790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.225648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.119771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.155191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.057407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.068617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.978037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.900696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.769318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.685102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.613306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.625430] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5739.640306] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5739.648524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.790261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.860470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.802032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.853907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.748098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.583977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.522275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.340149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.178255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.902688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.090760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.995842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.032955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.061981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.915321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.761906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.730077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.593480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.572330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.112771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.181797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.243064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.179668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.225474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.190813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.334417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.310031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.107083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.264718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.148512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.254053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.146659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.184214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.083548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.121553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.101594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.193782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.141357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.120924] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5778.129721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.143435] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5778.954938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.127951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.918041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.142316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.136226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.053902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.054143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.100208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.083961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.001544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.812402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.799468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.562816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.394884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.200211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.160529] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5793.172268] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5793.189121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.013652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.808811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.674440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.557937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.255293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.270542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.227503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.066232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.990773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.972217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.901951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.807677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.771490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.612767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.376252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.243947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.174733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.173310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.096345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.054563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.962508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.059404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.029424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.856485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.747893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.550847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.732740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.729241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.512863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.243557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.415291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.406143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.263227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.229538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.144626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.205821] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5826.229944] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5826.246919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.233654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.597527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.613697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.772836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.847898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.670135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.781926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.604449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.531422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.620987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.765687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.807969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.945647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.034502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.246188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.489497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.364832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.380671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.616934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.806541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.678328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.839737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.595557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.452348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.172714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.140331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.970402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.930795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.659718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.456719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.373172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.400654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.276783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.079681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.234930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.530179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.598687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.536762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.418825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.508217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.368538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.326982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.128580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.974979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.852718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.890126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.803777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.813707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.911912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.820914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.141587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.360776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.417799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.533416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.342866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.286148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.224036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.045949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.113533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.224356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.629139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.647248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.546442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.703966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.692226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.605189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.495203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.424743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.267272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.252971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.131270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.156192] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5898.173159] LustreError: 172071:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5898.181285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.251972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.215055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.097351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.029824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.130983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.040310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.982187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.849220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.946477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.056859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.915960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.942685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.737777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.621211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.504065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.428748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.377874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.221540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.213875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.981791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.957980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.714920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.947311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.147628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.219212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.241947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.303592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.341609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.503416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.634352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.565273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.365910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.456503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.114337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.024058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.984071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.129476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.134459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.139422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.071400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.047801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.921935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.974898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.934173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.914527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.814155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.061029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.014237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.199926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.100253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.000039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.983073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.077641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.014904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.915103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.842630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.907866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.981761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.965474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.013334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.966542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.963093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.819413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.735690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.509445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.373563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.137128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.013540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.914870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.672537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.516327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.427813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.256567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.034374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.874553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.839670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.538377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.504512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.392237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.341538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.080940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.998530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.859564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.711875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.570868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.304919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.112995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.845818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.687388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.417969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.268500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.091654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.875198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.660364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.421547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.164661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.935573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.719437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.563394] LustreError: 173871:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6326.588737] LustreError: 172071:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6792.167174] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6792.169627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6792.184286] Lustre: Skipped 460 previous similar messages [ 6792.195913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6792.695953] Lustre: server umount lustre-OST0000 complete [ 6798.818525] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6798.833301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6803.182231] Lustre: server umount lustre-MDT0000 complete [ 6804.454063] LustreError: 174180:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6804.485671] LustreError: 174180:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6806.925864] LustreError: 225896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788171202 with bad export cookie 5091290632909633404 [ 6806.936432] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6806.940685] LustreError: 225896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6817.422658] Lustre: server umount lustre-MDT0001 complete [ 6829.478123] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 6831.959693] Key type lgssc unregistered [ 6832.281077] LNet: 227282:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6832.289104] LNetError: 227282:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6832.315909] LNet: Removed LNI 192.168.202.114@tcp [ 6833.160869] Key type .llcrypt unregistered [ 6833.163249] Key type ._llcrypt unregistered [ 6858.730399] Key type ._llcrypt registered [ 6858.735180] Key type .llcrypt registered [ 6858.913671] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 6860.134323] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6860.396575] alg: No test for adler32 (adler32-zlib) [ 6861.527316] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 6861.825804] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 6863.554382] Key type lgssc registered [ 6864.906301] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6873.541118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6873.556539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6875.082168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6880.190994] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 6885.347119] LustreError: 228697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6885.368975] LustreError: 228697:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6889.215155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6889.589431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6895.623488] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 6904.363561] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6911.846153] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6919.347786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6919.932638] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6920.967170] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6927.773139] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 6937.666852] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6943.154705] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6944.932772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6944.945744] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6944.963582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.976936] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6944.977245] LustreError: 228717:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6949.834767] Lustre: lustre-OST0000: Not available for connect from 192.168.202.14@tcp (stopping) [ 6949.861777] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6949.882624] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6951.907060] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6951.908941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6951.940872] Lustre: Skipped 2 previous similar messages [ 6953.502448] Lustre: server umount lustre-OST0000 complete [ 6964.999206] LustreError: 228717:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6989.816153] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6990.127727] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6992.174920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.207918] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6996.966745] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7002.494636] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.509478] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7005.423970] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7018.594135] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 7024.055462] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9093984cf000.ost_server_uuid 50 [ 7025.604516] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9093984cf000.ost_server_uuid in FULL state after 0 sec [ 7030.761623] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7030.766929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7030.796690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7032.138923] Lustre: server umount lustre-OST0000 complete [ 7037.920725] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7037.933803] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7037.960100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7043.552944] LustreError: 230132:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7043.580698] LustreError: 230132:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7044.959816] Lustre: server umount lustre-MDT0000 complete [ 7048.676084] LustreError: 228698:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7048.694767] LustreError: 228698:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7048.976644] LustreError: 228677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788171444 with bad export cookie 15991126908286401281 [ 7048.992127] LustreError: 228677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7048.993738] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7059.694348] Lustre: server umount lustre-MDT0001 complete [ 7069.950278] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 7073.206403] Key type lgssc unregistered [ 7073.563687] LNet: 233931:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7073.568448] LNetError: 233931:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7074.599173] LNet: Removed LNI 192.168.202.114@tcp [ 7075.829896] Key type .llcrypt unregistered [ 7075.834716] Key type ._llcrypt unregistered [ 7114.245850] Key type ._llcrypt registered [ 7114.254343] Key type .llcrypt registered [ 7114.393832] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 7115.681500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7115.722333] alg: No test for adler32 (adler32-zlib) [ 7116.926589] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 7117.377968] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 7119.231569] Key type lgssc registered [ 7120.606715] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7131.548704] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7131.563900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7133.157384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7141.055125] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7143.394906] LustreError: 235665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7143.423051] LustreError: 235665:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7148.529191] LustreError: 235666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7152.601279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7153.323662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7160.856388] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7163.664722] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7171.013965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7171.714269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7177.910965] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7179.052790] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7188.137710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7188.240490] Lustre: lustre-OST0001: new disk, initializing [ 7188.243286] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7188.248708] Lustre: 237978:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7188.358487] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7194.556084] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7196.781419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7196.787928] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7196.864703] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7206.112728] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7217.206909] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7232.998566] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7233.002497] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7233.013845] Lustre: Skipped 2 previous similar messages [ 7233.037644] Lustre: Skipped 3 previous similar messages [ 7238.040710] Lustre: server umount lustre-MDT0000 complete [ 7238.125864] LustreError: 238439:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7238.143883] LustreError: 238439:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7241.639985] LustreError: 236403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788171636 with bad export cookie 14494086524045455286 [ 7241.650888] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7241.905869] Lustre: server umount lustre-MDT0001 complete [ 7255.105186] Lustre: server umount lustre-OST0000 complete [ 7259.108373] Lustre: 234927:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788171638/real 1788171638] req@ffff90c08a104a80 x1875033731460352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788171654 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7259.128980] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7259.139084] Lustre: Skipped 1 previous similar message [ 7259.381964] Lustre: server umount lustre-OST0001 complete [ 7281.370687] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 7291.358098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7291.909670] LustreError: 241085:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7291.994334] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7296.742116] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7297.001559] LustreError: 241086:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7305.060389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7305.441023] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7310.094366] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7312.892793] Lustre: 242227:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7319.243204] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7325.591901] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7330.822916] LustreError: 242579:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7330.865031] LustreError: 242579:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7334.037403] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7334.343359] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7334.351539] Lustre: Skipped 1 previous similar message [ 7337.458542] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7337.481768] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7340.945896] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7348.142975] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7351.405848] Lustre: 244097:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7362.016374] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7362.022935] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7362.035691] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7363.056411] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7363.062226] Lustre: Skipped 3 previous similar messages [ 7367.943184] Lustre: server umount mylustre-MDT0000 complete [ 7368.171406] LustreError: 241081:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7368.200584] LustreError: 241081:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7371.747901] LustreError: 241061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788171766 with bad export cookie 14494086524045458093 [ 7371.749958] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7371.768933] LustreError: 241061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7372.111107] Lustre: server umount mylustre-MDT0001 complete [ 7386.560507] Lustre: server umount mylustre-OST0000 complete [ 7389.663828] Lustre: 234926:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788171768/real 1788171768] req@ffff90c0858c3480 x1875033731518208/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788171784 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7389.700558] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7389.715295] Lustre: Skipped 3 previous similar messages [ 7390.726090] Lustre: server umount mylustre-OST0001 complete [ 7412.016216] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 7422.267726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7422.572448] LustreError: 246403:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7422.655116] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7426.682803] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7435.126466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7439.531434] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7442.650808] Lustre: 247547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7449.194918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7457.013468] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7466.219619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7467.505094] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7467.539094] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7473.054652] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7480.660684] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7484.523029] Lustre: 249422:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7492.064723] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7492.069677] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7492.079427] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7494.152094] Lustre: server umount tfs-MDT0000 complete [ 7497.312506] LustreError: 246381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788171892 with bad export cookie 14494086524045459983 [ 7497.315247] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7497.319128] LustreError: 246381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7497.609689] Lustre: server umount tfs-MDT0001 complete [ 7511.498623] Lustre: server umount tfs-OST0000 complete [ 7514.404584] Lustre: 234928:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788171893/real 1788171893] req@ffff90c0a7b53b80 x1875033731567360/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788171909 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7514.439208] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7514.447439] Lustre: Skipped 3 previous similar messages [ 7516.495406] Lustre: server umount tfs-OST0001 complete [ 7544.766642] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 7554.329109] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7554.692243] LustreError: 251670:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7554.713789] LustreError: 251670:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7554.809597] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7554.818886] Lustre: Skipped 3 previous similar messages [ 7558.951688] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7567.367696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7572.080932] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7575.041551] Lustre: 252813:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7581.310347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7588.515029] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7597.152422] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7599.478896] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7599.487338] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7602.788554] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7610.928940] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7614.227198] Lustre: 254683:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7620.245588] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:26:54 (1788172014) [ 7639.532059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7639.540073] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7639.557094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7639.560680] Lustre: Skipped 4 previous similar messages [ 7642.446285] Lustre: server umount lustre-MDT0000 complete [ 7645.550508] LustreError: 251645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788172040 with bad export cookie 14494086524045461649 [ 7645.557126] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7645.566674] LustreError: 251645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7650.788730] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7650.791836] Lustre: Skipped 6 previous similar messages [ 7651.908726] Lustre: server umount lustre-MDT0001 complete [ 7661.170991] Lustre: server umount lustre-OST0000 complete [ 7671.140463] Lustre: server umount lustre-OST0001 complete [ 7677.081894] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 7687.188090] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 7721.020616] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7721.020616] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7721.020616] [ 7721.040288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7721.355916] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7721.399482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7721.478993] Lustre: lustre-MDT0000: new disk, initializing [ 7721.551398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7721.557979] Lustre: Skipped 3 previous similar messages [ 7721.576121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7726.184637] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7736.931070] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7736.931070] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7736.931070] [ 7736.965360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7737.118519] Lustre: 257865:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7737.206547] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7737.210733] Lustre: Skipped 1 previous similar message [ 7737.282921] Lustre: lustre-MDT0001: new disk, initializing [ 7737.408377] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7737.416467] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7742.190433] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7752.700661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7752.984958] Lustre: lustre-OST0000: new disk, initializing [ 7752.987912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7752.996128] Lustre: 259447:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7754.972038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7755.000033] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7755.148068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7758.908433] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7771.512619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7771.688636] Lustre: lustre-OST0001: new disk, initializing [ 7771.691946] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7771.700705] Lustre: 260317:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7773.554738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7773.570153] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7773.723491] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7779.558664] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 7790.619853] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:29:44 (1788172184) [ 7804.677676] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:29:58 (1788172198) [ 7819.753098] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7819.767158] Lustre: Skipped 5 previous similar messages [ 7819.773144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7819.782207] Lustre: Skipped 1 previous similar message [ 7834.079129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7834.334732] Lustre: server umount lustre-OST0000 complete [ 7835.118491] LustreError: 259431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7835.143520] LustreError: 259431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 7839.716100] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7844.830322] Lustre: server umount lustre-MDT0000 complete [ 7848.715203] LustreError: 257857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788172243 with bad export cookie 14494086524045463203 [ 7848.720225] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7848.732833] LustreError: 257857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7853.475392] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7853.484307] Lustre: Skipped 9 previous similar messages [ 7863.263125] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7863.477170] Lustre: server umount lustre-MDT0001 complete [ 7873.472131] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 7946.221484] Lustre: server umount lustre-OST0001 complete [ 7953.473496] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 7969.113859] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8007.605278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8007.940460] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8007.973052] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8008.046828] Lustre: lustre-MDT0000: new disk, initializing [ 8008.156864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8008.159841] Lustre: Skipped 3 previous similar messages [ 8008.172438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8013.326679] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8024.604183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8024.828580] Lustre: 265117:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8024.939609] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8024.943628] Lustre: Skipped 1 previous similar message [ 8025.064751] Lustre: lustre-MDT0001: new disk, initializing [ 8025.308045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8025.332188] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8029.849632] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8039.693188] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8045.067482] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8052.615035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8052.935173] Lustre: 267085:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8054.855032] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8055.055715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8060.658565] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8070.877452] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8080.872662] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8080.885361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8080.891943] LustreError: Skipped 1 previous similar message [ 8080.892957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8080.892962] Lustre: Skipped 1 previous similar message [ 8080.944182] Lustre: Skipped 6 previous similar messages [ 8083.449416] Lustre: server umount lustre-OST0000 complete [ 8093.810803] Lustre: server umount lustre-MDT0000 complete [ 8096.226842] LustreError: 265127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8096.259379] LustreError: 265127:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 8097.622488] LustreError: 265108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788172492 with bad export cookie 14494086524045464673 [ 8097.628047] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8097.633676] LustreError: 265108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8098.152052] Lustre: server umount lustre-MDT0001 complete [ 8114.029309] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 8118.412495] Key type lgssc unregistered [ 8118.847795] LNet: 268847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8118.857417] LNetError: 268847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8118.873719] LNet: Removed LNI 192.168.202.114@tcp [ 8120.096152] Key type .llcrypt unregistered [ 8120.097878] Key type ._llcrypt unregistered [ 8152.230260] Key type ._llcrypt registered [ 8152.232641] Key type .llcrypt registered [ 8152.350233] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8153.654507] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8153.698961] alg: No test for adler32 (adler32-zlib) [ 8154.742357] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 8154.924546] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 8156.607894] Key type lgssc registered [ 8158.031441] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8166.263502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8166.289256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8167.748118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8172.024771] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8177.124477] LustreError: 270329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8177.143021] LustreError: 270329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8180.549453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8180.900591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8185.961712] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8194.157364] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8200.847423] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8208.036552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8208.526542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8212.655467] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8214.672550] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8222.557281] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8223.627535] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8228.320215] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8228.328401] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8228.340479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8230.369836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8230.382606] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8230.382875] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8231.709322] Lustre: server umount lustre-OST0000 complete [ 8237.028618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8237.046807] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8237.063640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8237.076261] Lustre: Skipped 1 previous similar message [ 8240.609952] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8240.611530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8242.759703] Lustre: server umount lustre-MDT0000 complete [ 8245.749486] LustreError: 270330:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8245.781085] LustreError: 270330:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8248.129777] LustreError: 272859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788172643 with bad export cookie 4568493290716731975 [ 8248.143388] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8248.146571] LustreError: 272859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8249.038823] Lustre: server umount lustre-MDT0001 complete [ 8260.925268] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 8265.058975] Key type lgssc unregistered [ 8265.460847] LNet: 273918:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8265.469171] LNetError: 273918:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8265.506601] LNet: Removed LNI 192.168.202.114@tcp [ 8266.696774] Key type .llcrypt unregistered [ 8266.705913] Key type ._llcrypt unregistered [ 8283.844090] Key type ._llcrypt registered [ 8283.846949] Key type .llcrypt registered [ 8284.004599] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8285.786655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8285.847956] alg: No test for adler32 (adler32-zlib) [ 8287.066644] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 8287.396055] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 8289.151150] Key type lgssc registered [ 8290.765940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8299.845978] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8299.877739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8315.551478] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8320.671436] LustreError: 275112:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.114@tcp: failed processing log, type 4: rc = -110 [ 8347.295278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8352.750407] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8359.404455] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8361.386332] Lustre: server umount lustre-OST0000 complete [ 8376.919793] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 8379.308271] Key type lgssc unregistered [ 8379.533944] LNet: 276744:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8379.539432] LNetError: 276744:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8380.582624] LNet: Removed LNI 192.168.202.114@tcp [ 8381.474847] Key type .llcrypt unregistered [ 8381.476783] Key type ._llcrypt unregistered [ 8419.264641] Key type ._llcrypt registered [ 8419.266981] Key type .llcrypt registered [ 8419.367894] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8420.906840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8420.966738] alg: No test for adler32 (adler32-zlib) [ 8422.011628] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 8422.243401] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 8423.969608] Key type lgssc registered [ 8425.252866] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8441.412904] loop0: detected capacity change from 0 to 204800000 [ 8441.790696] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8441.802666] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8442.235379] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8443.753317] loop0: detected capacity change from 0 to 204800000 [ 8444.003183] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.038235] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.663819] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.172436] loop0: detected capacity change from 0 to 204800000 [ 8446.523913] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.537059] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.793622] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8447.997433] loop0: detected capacity change from 0 to 204800000 [ 8448.200037] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8448.228340] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8448.361237] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8449.792117] loop0: detected capacity change from 0 to 204800000 [ 8449.826656] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8454.859421] loop0: detected capacity change from 0 to 204800000 [ 8454.922614] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8459.590726] loop0: detected capacity change from 0 to 204800000 [ 8459.624618] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8464.413333] loop0: detected capacity change from 0 to 204800000 [ 8464.449538] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8468.705383] loop0: detected capacity change from 0 to 204800000 [ 8468.840756] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8468.929510] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8469.080070] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8469.090495] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8469.104452] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8469.121453] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8469.136966] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8469.154686] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8469.169304] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8469.193894] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8469.209464] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8469.224242] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8469.238283] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8469.263502] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8469.271967] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8469.284835] Lustre: 280159:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8475.381293] Lustre: server umount MGS complete [ 8476.319519] loop0: detected capacity change from 0 to 204800000 [ 8476.390946] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8477.004820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8477.062319] Lustre: lustre-MDT0000: Aborting client recovery [ 8477.894236] loop1: detected capacity change from 0 to 204800000 [ 8478.157373] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8478.232696] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8478.241637] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8478.249587] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8478.256685] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8478.262815] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8478.272288] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8478.285633] Lustre: 280590:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8478.306615] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8478.457349] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8478.503343] Lustre: lustre-MDT0001: Aborting client recovery [ 8479.379138] loop2: detected capacity change from 0 to 204800000 [ 8479.580105] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8479.652359] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8479.664347] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8479.671164] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8479.678960] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8479.687114] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8479.693931] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8479.700545] Lustre: 280670:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8479.768673] Lustre: Found index 0 for lustre-OST0000, updating log [ 8479.795673] Lustre: 280688:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8479.864240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8479.882971] Lustre: lustre-OST0000: Aborting recovery [ 8480.682802] loop3: detected capacity change from 0 to 204800000 [ 8480.863294] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8480.980465] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8480.988360] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8480.996535] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8481.003865] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8481.010514] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8481.017953] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8481.026167] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8481.053650] Lustre: Found index 1 for lustre-OST0001, updating log [ 8481.076030] Lustre: 280759:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8481.159528] Lustre: lustre-OST0001: Aborting recovery [ 8486.404023] LustreError: 280534:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8487.421386] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8487.470919] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8487.490052] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8487.505025] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8488.423498] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8488.464040] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8489.076817] LustreError: 280501:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 555us: rc = -2 [ 8489.605991] LustreError: 280502:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 673us: rc = -2 [ 8489.620109] LustreError: 280502:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 8503.780055] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8503.791936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8503.819339] Lustre: Skipped 1 previous similar message [ 8503.840779] Lustre: Skipped 3 previous similar messages [ 8508.846131] Lustre: server umount lustre-MDT0000 complete [ 8508.916271] LustreError: 280503:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8508.960459] LustreError: 280503:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 8510.102572] LustreError: 280483:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788172905 with bad export cookie 16230068129904510679 [ 8510.115384] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8510.119117] LustreError: 280483:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8514.016765] LustreError: 280498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8514.018469] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8514.020705] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8514.046082] LustreError: 280498:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8514.052954] Lustre: Skipped 3 previous similar messages [ 8515.043379] LustreError: 280500:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8515.048218] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8515.071829] LustreError: 280500:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8515.099969] Lustre: Skipped 2 previous similar messages [ 8516.766526] Lustre: server umount lustre-MDT0001 complete [ 8523.776995] Lustre: server umount lustre-OST0000 complete [ 8531.693369] Lustre: server umount lustre-OST0001 complete [ 8541.512417] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:42:15 (1788172935) [ 8569.644645] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 8580.596427] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8621.511145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8621.884090] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8621.947051] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8622.211054] Lustre: lustre-MDT0000: new disk, initializing [ 8622.412028] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8622.426257] Lustre: Skipped 1 previous similar message [ 8622.443577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8628.665492] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8638.768420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8638.914591] Lustre: 284832:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8638.984640] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8638.987434] Lustre: Skipped 1 previous similar message [ 8639.058464] Lustre: lustre-MDT0001: new disk, initializing [ 8639.153395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8639.183348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8639.201864] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8644.003229] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8654.064726] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8659.732323] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8666.735370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8667.144707] Lustre: lustre-OST0000: new disk, initializing [ 8667.150426] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8667.162439] Lustre: 286801:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8667.319784] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8673.383884] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8673.877405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8673.892102] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8674.018016] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8682.350228] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8687.308988] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8696.815780] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8705.968494] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8756.197105] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8756.200372] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8756.225539] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8756.259040] Lustre: Skipped 1 previous similar message [ 8757.504569] Lustre: server umount lustre-OST0000 complete [ 8761.312727] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8761.334931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8761.341829] Lustre: Skipped 2 previous similar messages [ 8767.232061] Lustre: server umount lustre-MDT0000 complete [ 8770.739605] LustreError: 286800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788173165 with bad export cookie 16230068129904513409 [ 8770.749887] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8770.758792] LustreError: 286800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8771.297512] Lustre: server umount lustre-MDT0001 complete [ 8779.929643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8784.238844] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8790.322783] Lustre: server umount MGS complete [ 8802.994811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8803.383473] LustreError: 290710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8803.410415] LustreError: 290723:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8803.458241] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8808.107261] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8808.929737] LustreError: 290711:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8815.991776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8821.219541] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8827.992687] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8833.727202] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8839.741342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8840.026062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8840.029949] Lustre: Skipped 1 previous similar message [ 8846.507565] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 8855.580244] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8882.144578] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8882.147546] LustreError: Skipped 1 previous similar message [ 8882.148944] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8882.168759] Lustre: Skipped 1 previous similar message [ 8882.176873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8882.179582] Lustre: Skipped 2 previous similar messages [ 8884.192474] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8886.496880] Lustre: server umount lustre-OST0000 complete [ 8893.410516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8893.416557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8893.443548] Lustre: Skipped 1 previous similar message [ 8896.137458] Lustre: server umount lustre-MDT0000 complete [ 8899.341266] LustreError: 290692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788173294 with bad export cookie 16230068129904515264 [ 8899.343720] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8899.359935] LustreError: 290692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8899.708205] Lustre: server umount lustre-MDT0001 complete [ 8912.743412] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 8915.637715] Key type lgssc unregistered [ 8915.949794] LNet: 294659:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8915.964326] LNetError: 294659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8917.034373] LNet: Removed LNI 192.168.202.114@tcp [ 8917.912481] Key type .llcrypt unregistered [ 8917.914404] Key type ._llcrypt unregistered [ 8952.245162] Key type ._llcrypt registered [ 8952.251102] Key type .llcrypt registered [ 8952.375845] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 8966.640689] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 8967.345458] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8967.360957] alg: No test for adler32 (adler32-zlib) [ 8968.550098] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 8968.794746] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 8970.441658] Key type lgssc registered [ 8971.802679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9004.212078] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9004.243789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9005.474826] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9005.498799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9005.568758] Lustre: lustre-MDT0000: new disk, initializing [ 9005.635192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9005.654894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9010.021376] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9020.810167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9020.978911] Lustre: 298615:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9021.092807] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9021.100121] Lustre: Skipped 1 previous similar message [ 9021.187901] Lustre: lustre-MDT0001: new disk, initializing [ 9021.282552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9021.312620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9021.317679] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9025.315776] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9033.951870] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9039.281922] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9045.139558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9045.371612] Lustre: lustre-OST0000: new disk, initializing [ 9045.375914] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9045.381896] Lustre: 300585:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9045.477996] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9052.202680] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9054.286393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9054.299451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9054.427425] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9061.194722] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9067.267934] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9080.191580] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9087.406344] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9094.989277] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9126.368155] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9126.377356] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9126.403162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9126.413119] Lustre: Skipped 1 previous similar message [ 9131.490454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9131.501881] Lustre: Skipped 1 previous similar message [ 9136.611956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9136.621545] Lustre: Skipped 1 previous similar message [ 9140.191150] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9140.491183] Lustre: server umount lustre-OST0000 complete [ 9144.295438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9144.304983] LustreError: Skipped 1 previous similar message [ 9144.307861] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9144.335425] Lustre: Skipped 1 previous similar message [ 9144.359050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9146.850356] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9150.238035] Lustre: server umount lustre-MDT0000 complete [ 9153.977778] LustreError: 299411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788173549 with bad export cookie 16169493379761309124 [ 9153.983372] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9154.323755] Lustre: server umount lustre-MDT0001 complete [ 9162.323706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9166.369701] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9171.092583] Lustre: server umount MGS complete [ 9182.943783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9183.925128] LustreError: 304540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9183.969967] LustreError: 304540:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9183.990207] LustreError: 304553:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9184.101187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9189.346394] LustreError: 304541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9189.380932] LustreError: 304541:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9189.768344] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9194.477648] LustreError: 304540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9199.599903] LustreError: 304541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9201.495746] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9202.107464] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9207.809319] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9217.189511] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9223.157711] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9230.358973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9230.691253] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9237.771937] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9246.160868] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9294.304138] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9294.316434] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9294.350261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9294.362068] Lustre: Skipped 2 previous similar messages [ 9297.115062] Lustre: server umount lustre-OST0000 complete [ 9304.548796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9304.567091] LustreError: Skipped 1 previous similar message [ 9304.570383] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9304.576498] Lustre: Skipped 1 previous similar message [ 9304.581867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9304.585521] Lustre: Skipped 2 previous similar messages [ 9315.295148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9315.730289] Lustre: server umount lustre-MDT0000 complete [ 9315.813591] LustreError: 305488:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9320.786085] LustreError: 304522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788173715 with bad export cookie 16169493379761310895 [ 9320.794704] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9320.804953] LustreError: 304522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9321.343613] Lustre: server umount lustre-MDT0001 complete [ 9335.102201] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [ 9337.700337] Key type lgssc unregistered [ 9337.933500] LNet: 308514:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9337.936171] LNetError: 308514:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9337.952845] LNet: Removed LNI 192.168.202.114@tcp [ 9338.992169] Key type .llcrypt unregistered [ 9338.994130] Key type ._llcrypt unregistered [ 9360.400581] Key type ._llcrypt registered [ 9360.406593] Key type .llcrypt registered [ 9360.519897] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [ 9372.436940] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 9373.867584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9373.902613] alg: No test for adler32 (adler32-zlib) [ 9374.995632] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [ 9375.297896] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [ 9377.047186] Key type lgssc registered [ 9378.260235] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9401.268078] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [ 9417.070391] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9417.093513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9418.327240] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9418.356644] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9418.440885] Lustre: lustre-MDT0000: new disk, initializing [ 9418.515432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9418.534244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9422.798774] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9432.888664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9433.101552] Lustre: lustre-OST0000: new disk, initializing [ 9433.105324] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9433.110076] Lustre: Skipped 1 previous similar message [ 9433.125388] Lustre: 312544:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9433.178175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9439.093518] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [ 9443.396757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9443.407922] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9443.442860] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10180.578123] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10180.589788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10185.834672] Lustre: server umount lustre-MDT0000 complete [10189.987829] LustreError: 311731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788174585 with bad export cookie 7664567549073949883 [10190.009170] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10190.137651] Lustre: server umount lustre-OST0000 complete [10217.185987] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [10232.114270] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [10276.320813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10276.645664] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10276.689601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10276.837950] Lustre: lustre-MDT0000: new disk, initializing [10276.939342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10276.959577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10281.584673] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10293.055405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10293.180353] Lustre: 316897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10293.229323] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10293.232901] Lustre: Skipped 1 previous similar message [10293.365832] Lustre: lustre-MDT0001: new disk, initializing [10293.479293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10293.529636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10293.550674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10298.816216] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10309.089520] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10316.592226] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10323.976123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10324.314535] Lustre: lustre-OST0000: new disk, initializing [10324.318546] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10324.327943] Lustre: 318868:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10324.428044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10330.772583] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10331.200363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10331.217994] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10331.320300] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10340.758808] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10345.082268] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10345.273252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10350.275466] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10350.521390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10357.216400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10357.219767] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10357.226874] LustreError: Skipped 1 previous similar message [10357.227666] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10357.248482] Lustre: Skipped 1 previous similar message [10358.610299] Lustre: server umount lustre-OST0000 complete [10362.336656] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10362.337582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10362.352902] Lustre: Skipped 1 previous similar message [10362.360497] Lustre: Skipped 2 previous similar messages [10367.456935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10367.462154] Lustre: Skipped 1 previous similar message [10368.858473] Lustre: server umount lustre-MDT0000 complete [10372.577081] LustreError: 316910:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10372.586255] LustreError: 316910:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10372.912617] LustreError: 316890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788174768 with bad export cookie 7664567549076890695 [10372.913356] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10372.933200] LustreError: 316890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10373.356651] Lustre: server umount lustre-MDT0001 complete [10377.049584] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10378.828969] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:12:52 (1788174772) [10387.203274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10387.624582] LustreError: 321185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10387.692888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10392.339134] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10393.061545] LustreError: 321184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10397.152951] LustreError: 321185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10401.964993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10402.331699] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10407.242828] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10415.634813] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10422.399314] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10429.024877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10429.298810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10434.544341] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10443.164766] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10449.638406] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10449.869475] Lustre: lustre-OST0001: new disk, initializing [10449.873077] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10449.892586] Lustre: 324020:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10450.002931] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10457.977753] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10459.833699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10459.846081] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10459.925703] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10460.966925] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10460.982909] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10462.071715] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10462.093934] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10464.222942] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10464.259699] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10468.328229] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10468.357090] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10471.241525] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10471.524239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10476.598583] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10476.635988] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10480.355335] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff909398462800.ost_server_uuid 50 [10482.292230] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff909398462800.ost_server_uuid in FULL state after 0 sec [10490.472339] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff909398462800.ost_server_uuid 50 [10492.607469] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff909398462800.ost_server_uuid in IDLE state after 0 sec [10493.091340] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10493.110203] LustreError: 324453:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 30 previous similar messages [10495.239016] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10510.799955] Lustre: lustre-OST0001: Not available for connect from 192.168.202.14@tcp (stopping) [10510.818070] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10510.823211] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10510.825719] LustreError: Skipped 1 previous similar message [10510.856281] Lustre: Skipped 1 previous similar message [10515.881278] LustreError: 323007:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10516.052627] Lustre: server umount lustre-OST0001 complete [10524.642662] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10524.644716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10524.655380] LustreError: Skipped 1 previous similar message [10524.656919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10524.656926] Lustre: Skipped 4 previous similar messages [10524.710680] Lustre: Skipped 1 previous similar message [10526.695340] LustreError: 323007:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10526.717611] LustreError: 323007:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [10528.779911] Lustre: server umount lustre-OST0000 complete [10535.392122] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10535.397438] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10539.249681] Lustre: server umount lustre-MDT0000 complete [10543.249086] LustreError: 321165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788174938 with bad export cookie 7664567549076891584 [10543.254638] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10543.267163] LustreError: 321165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10543.628507] Lustre: server umount lustre-MDT0001 complete [10554.514504] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [10557.841910] Key type lgssc unregistered [10558.191282] LNet: 326714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10558.198623] LNetError: 326714:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10558.237650] LNet: Removed LNI 192.168.202.114@tcp [10559.768319] Key type .llcrypt unregistered [10559.770061] Key type ._llcrypt unregistered [10583.791793] Key type ._llcrypt registered [10583.796282] Key type .llcrypt registered [10583.894033] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [10585.165769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10585.431395] alg: No test for adler32 (adler32-zlib) [10586.507430] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [10586.703827] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [10588.362077] Key type lgssc registered [10589.688232] Lustre: Echo OBD driver; http://www.lustre.org/ [10597.865169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10597.886588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10599.430233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10605.401453] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10609.635982] LustreError: 328137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10609.650613] LustreError: 328137:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10614.467906] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10614.758127] LustreError: 328138:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10619.877740] LustreError: 328137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10622.739441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10623.085797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10626.822711] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10636.700593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10637.016485] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10643.310185] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10644.205864] LustreError: 329818:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10644.238109] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10648.305807] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10648.341632] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10650.479641] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10657.665614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10657.864021] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10662.919937] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10663.664536] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10671.066317] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10678.032958] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10678.229079] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10780.648997] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10780.663151] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10780.668423] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10780.686388] Lustre: Skipped 1 previous similar message [10783.421914] Lustre: server umount lustre-OST0001 complete [10785.765378] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10785.771729] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10790.881137] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10790.891623] Lustre: Skipped 1 previous similar message [10790.899358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10790.904477] Lustre: Skipped 1 previous similar message [10794.649199] Lustre: server umount lustre-OST0000 complete [10801.120858] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10801.122984] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10801.134577] Lustre: Skipped 2 previous similar messages [10801.143262] Lustre: Skipped 1 previous similar message [10805.004988] Lustre: server umount lustre-MDT0000 complete [10806.244780] LustreError: 331822:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10806.260956] LustreError: 331822:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10808.225039] LustreError: 331823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788175203 with bad export cookie 9537443130791510380 [10808.233025] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10808.237543] LustreError: 331823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10808.816388] Lustre: server umount lustre-MDT0001 complete [10820.323932] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [10823.771724] Key type lgssc unregistered [10824.094244] LNet: 333081:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10824.103127] LNetError: 333081:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10824.119651] LNet: Removed LNI 192.168.202.114@tcp [10825.183135] Key type .llcrypt unregistered [10825.187547] Key type ._llcrypt unregistered [10876.090653] Key type ._llcrypt registered [10876.092893] Key type .llcrypt registered [10876.209207] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [10877.407792] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10877.605631] alg: No test for adler32 (adler32-zlib) [10878.701047] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [10878.938330] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [10880.751182] Key type lgssc registered [10882.300717] Lustre: Echo OBD driver; http://www.lustre.org/ [10893.316664] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10893.344290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10894.728198] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10899.249748] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10904.038891] LustreError: 335374:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10904.058504] LustreError: 335374:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10909.180366] LustreError: 335375:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10909.660643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10910.052546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10915.328609] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10919.436400] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10926.876328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10927.526439] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10934.280880] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10935.748132] LustreError: 336862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10938.879970] LustreError: 336864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10938.928396] LustreError: 336864:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10938.941708] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10943.973958] LustreError: 336863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10944.001322] LustreError: 336863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10944.681357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10945.342533] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10950.665456] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10953.346267] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [10964.124816] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10973.668972] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10991.587900] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10991.593904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10991.610554] Lustre: Skipped 2 previous similar messages [10991.617607] Lustre: Skipped 1 previous similar message [10993.983229] Lustre: server umount lustre-MDT0000 complete [10996.712818] LustreError: 335370:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10996.721465] LustreError: 335370:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10997.432539] LustreError: 335355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788175392 with bad export cookie 2453657852347741214 [10997.439501] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10997.450451] LustreError: 335355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10997.734453] Lustre: server umount lustre-MDT0001 complete [11011.141379] Lustre: server umount lustre-OST0000 complete [11025.440425] Lustre: server umount lustre-OST0001 complete [11042.657216] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [11054.123313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11054.414460] LustreError: 340642:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11054.423760] LustreError: 340642:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11054.477075] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11058.843700] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11067.584477] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11067.910799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11072.230498] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11075.466154] Lustre: 341782:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11081.659516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11088.874630] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11093.293406] LustreError: 342137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11093.323639] LustreError: 342137:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11097.174957] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11097.401469] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11097.406812] Lustre: Skipped 1 previous similar message [11099.451912] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11100.531223] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11102.684229] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11109.787985] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11113.921732] Lustre: 343652:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11119.078265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11119.086659] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11119.104100] Lustre: Skipped 1 previous similar message [11119.114518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11119.122102] Lustre: Skipped 2 previous similar messages [11121.120282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11121.126751] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11121.136977] Lustre: Skipped 1 previous similar message [11123.985129] Lustre: server umount lustre-MDT0000 complete [11127.937497] LustreError: 340622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788175523 with bad export cookie 2453657852347742915 [11127.939825] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11127.947589] LustreError: 340622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11128.470706] Lustre: server umount lustre-MDT0001 complete [11142.640290] Lustre: server umount lustre-OST0000 complete [11156.483406] Lustre: server umount lustre-OST0001 complete [11160.604465] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11162.769313] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11165.143763] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:25:58 (1788175558) [11190.941382] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [11200.547562] loop0: detected capacity change from 0 to 1099511627776 [11201.228793] SGI XFS with ACLs, security attributes, quota, no debug enabled [11201.246359] *** VALIDATE xfs *** [11201.262637] XFS (loop0): Mounting V5 Filesystem [11201.290429] XFS (loop0): Ending clean mount [11202.701309] loop1: detected capacity change from 0 to 18691697672192 [11304.007286] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [11317.124797] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [11361.864184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11362.110161] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11362.133328] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11362.233206] Lustre: lustre-MDT0000: new disk, initializing [11362.311305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11362.327849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11367.139525] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11377.878484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11377.959831] Lustre: 348793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11377.995479] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11378.004551] Lustre: Skipped 1 previous similar message [11378.062310] Lustre: lustre-MDT0001: new disk, initializing [11378.207241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11378.224263] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11384.310783] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11394.890427] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11401.559717] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11409.891604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11410.277963] Lustre: lustre-OST0000: new disk, initializing [11410.283541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11410.296676] Lustre: 350760:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11411.753794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11411.761202] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11411.930834] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11416.688740] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11425.738818] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11432.614372] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11432.948674] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11437.353402] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11437.540240] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11442.659353] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11442.667374] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11442.697612] Lustre: Skipped 1 previous similar message [11442.701866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11442.719282] Lustre: Skipped 4 previous similar messages [11445.527325] Lustre: server umount lustre-OST0000 complete [11449.838530] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11449.847152] LustreError: Skipped 1 previous similar message [11449.854367] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11449.866242] Lustre: Skipped 1 previous similar message [11449.870568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11454.939825] Lustre: server umount lustre-MDT0000 complete [11458.019200] LustreError: 348803:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11458.051420] LustreError: 348803:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11459.427146] LustreError: 348784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788175854 with bad export cookie 2453657852347744469 [11459.435240] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11459.443052] LustreError: 348784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11459.795105] Lustre: server umount lustre-MDT0001 complete [11471.551131] XFS (loop0): Unmounting Filesystem [11476.401921] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:31:10 (1788175870) [11483.799845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11484.375255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11484.381673] Lustre: Skipped 2 previous similar messages [11489.280778] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11498.274422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11503.459717] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11510.923401] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11517.240844] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11524.891218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11531.792606] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11538.574917] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11550.184182] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11550.205507] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11550.218207] Lustre: Skipped 1 previous similar message [11550.228246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11550.232618] Lustre: Skipped 2 previous similar messages [11554.276429] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11555.851969] Lustre: server umount lustre-OST0000 complete [11569.643244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11569.673774] Lustre: Skipped 6 previous similar messages [11574.239805] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11574.469740] Lustre: server umount lustre-MDT0000 complete [11581.410737] LustreError: 353304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788175976 with bad export cookie 2453657852347745344 [11581.412816] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11581.439358] LustreError: 353304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11582.540391] Lustre: server umount lustre-MDT0001 complete [11599.266529] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [11603.266261] Key type lgssc unregistered [11603.628782] LNet: 356959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11603.638076] LNetError: 356959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11603.686707] LNet: Removed LNI 192.168.202.114@tcp [11605.235309] Key type .llcrypt unregistered [11605.239110] Key type ._llcrypt unregistered [11636.439539] Key type ._llcrypt registered [11636.441930] Key type .llcrypt registered [11636.530819] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [11638.017421] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11638.098436] alg: No test for adler32 (adler32-zlib) [11639.218216] Lustre: Lustre: Build Version: 2.17.56_54_gb664824 [11639.505818] LNet: Added LNI 192.168.202.114@tcp [8/256/0/180] [11641.208924] Key type lgssc registered [11642.481570] Lustre: Echo OBD driver; http://www.lustre.org/ [11650.250890] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11650.263319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11651.866970] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11657.725672] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11661.283768] LustreError: 358529:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11661.319514] LustreError: 358529:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11665.846172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11666.178337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11671.786820] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11681.614456] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11690.668929] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11701.404421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11701.859487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11710.623547] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11721.390480] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11728.921120] Lustre: server umount lustre-MDT0001 complete [11729.331986] LustreError: 358524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11731.944627] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11731.957148] Lustre: Skipped 1 previous similar message [11731.962044] LustreError: 359498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11731.970310] LustreError: 359498:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11734.440691] LustreError: 358524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11739.570755] LustreError: 358524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11739.586300] LustreError: 358524:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11742.642123] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11742.693485] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11742.695534] Lustre: Found index 1 for lustre-MDT0001, updating log [11742.728869] Lustre: 358517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11743.031068] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11747.150862] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [11748.328534] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11750.904501] LustreError: 362120:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11750.912919] LustreError: 362120:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11750.930909] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11750.991383] LustreError: 362127:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.114@tcp: cfg command failed: rc = -17 [11751.003597] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11751.003597] [12053.476268] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12053.494362] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12053.503233] Lustre: Skipped 1 previous similar message [12057.059912] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.14@tcp (stopping) [12058.594398] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12058.955962] Lustre: server umount lustre-MDT0001 complete [12062.126867] LustreError: 358525:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.14@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12062.148028] LustreError: 358525:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12068.332195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12068.420606] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12068.424637] Lustre: Found index 1 for lustre-MDT0001, updating log [12068.453658] Lustre: 358517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12068.724716] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12073.557982] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [12073.980322] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12076.549755] LustreError: 370575:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12076.575120] LustreError: 370575:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12076.593780] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12076.602654] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12076.696368] LustreError: 370583:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12076.714065] LustreError: 370583:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12076.720160] LustreError: 370583:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.114@tcp: cfg command failed: rc = -17 [12076.729802] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.114@tcp [12076.729802] [12379.601755] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.14@tcp (stopping) [12379.609646] Lustre: Skipped 1 previous similar message [12381.157832] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12381.161424] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12381.174657] Lustre: Skipped 1 previous similar message [12384.682933] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.14@tcp (stopping) [12384.701850] Lustre: Skipped 2 previous similar messages [12385.908356] Lustre: server umount lustre-MDT0001 complete [12386.272482] LustreError: 358528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12386.302811] LustreError: 358528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12397.059223] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12397.146223] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12397.153166] Lustre: Found index 1 for lustre-MDT0001, updating log [12397.196532] Lustre: 358515:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12397.569993] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12402.541683] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [12402.695386] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12405.772106] LustreError: 378986:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12405.780335] LustreError: 378986:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12405.792692] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12405.801480] LustreError: 358520:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12405.913052] LustreError: 378989:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12405.942684] LustreError: 378989:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.114@tcp: cfg command failed: rc = -114 [12405.962101] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12405.962101] [12730.339771] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12730.345986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12730.357548] Lustre: Skipped 3 previous similar messages [12730.363026] Lustre: Skipped 1 previous similar message [12732.096245] Lustre: server umount lustre-MDT0000 complete [12735.467351] LustreError: 358528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12735.476774] LustreError: 358528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [12735.521748] LustreError: 358509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788177130 with bad export cookie 4644395911909232211 [12735.524522] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12735.530117] LustreError: 358509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12740.580685] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12742.106240] Lustre: server umount lustre-MDT0001 complete [12745.898164] Lustre: server umount lustre-OST0000 complete [12756.299623] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [12767.534535] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [12803.084599] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:53:17 (1788177197) [12810.988146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12811.406249] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12811.472543] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12811.701327] Lustre: lustre-MDT0000: new disk, initializing [12811.842840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12811.865287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12817.184128] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [12829.179584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12829.328699] Lustre: 389413:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12829.406725] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12829.411699] Lustre: Skipped 1 previous similar message [12829.573411] Lustre: lustre-MDT0001: new disk, initializing [12829.797935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12829.855728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12829.875613] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12835.940400] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [12847.533354] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12854.832699] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12863.349433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12863.629861] Lustre: lustre-OST0000: new disk, initializing [12863.634849] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12863.640535] Lustre: 391373:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12863.726555] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12869.913408] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_default_debug -1 all [12873.290770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12873.311774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12873.415570] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12880.062927] Lustre: DEBUG MARKER: oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12889.061469] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12889.063515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12889.073842] Lustre: Skipped 2 previous similar messages [12889.084300] Lustre: Skipped 4 previous similar messages [12889.840114] Lustre: server umount lustre-MDT0000 complete [12893.121647] LustreError: 390215:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12893.122498] LustreError: 389404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788177288 with bad export cookie 4644395911909234164 [12893.142372] LustreError: MGC192.168.202.114@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12893.475941] Lustre: server umount lustre-MDT0001 complete [12907.890286] Lustre: server umount lustre-OST0000 complete [12938.114969] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing set_hostid [12951.716367] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing load_modules_local [12988.754815] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12718 sec ========= 07:56:23 (1788177383) [12990.352265] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:56:24 (1788177384) === [12993.754477] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:56:27 (1788177387) === [13032.757749] Lustre: DEBUG MARKER: oleg214-server.virtnet: executing unload_modules_local [13035.273496] Key type lgssc unregistered [13035.525660] LNet: 397717:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13035.529655] LNetError: 397717:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13036.584758] LNet: Removed LNI 192.168.202.114@tcp [13037.365164] Key type .llcrypt unregistered [13037.367148] Key type ._llcrypt unregistered