[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 576641695 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002310] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.008028] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010123] LSM: Security Framework initializing [ 0.012036] Yama: becoming mindful. [ 0.013033] SELinux: Initializing. [ 0.014062] *** VALIDATE selinux *** [ 0.023189] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028417] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029135] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031094] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032113] *** VALIDATE tmpfs *** [ 0.034109] *** VALIDATE proc *** [ 0.035234] *** VALIDATE cgroup *** [ 0.036008] *** VALIDATE cgroup2 *** [ 0.038080] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039164] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041031] Spectre V2 : User space: Vulnerable [ 0.042010] Speculative Store Bypass: Vulnerable [ 0.045719] debug: unmapping init [mem 0xffffffffa7a59000-0xffffffffa7a60fff] [ 0.047530] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048785] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049025] ... version: 2 [ 0.050011] ... bit width: 48 [ 0.051010] ... generic registers: 4 [ 0.052011] ... value mask: 0000ffffffffffff [ 0.053014] ... max period: 00007fffffffffff [ 0.054014] ... fixed-purpose events: 3 [ 0.055013] ... event mask: 000000070000000f [ 0.056276] rcu: Hierarchical SRCU implementation. [ 0.058545] smp: Bringing up secondary CPUs ... [ 0.059792] x86: Booting SMP configuration: [ 0.060023] .... node #0, CPUs: #1 #2 #3 [ 0.068094] smp: Brought up 1 node, 4 CPUs [ 0.070013] smpboot: Max logical packages: 1 [ 0.071012] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.117942] node 0 deferred pages initialised in 45ms [ 0.122104] devtmpfs: initialized [ 0.124394] x86/mm: Memory block size: 128MB [ 0.127610] gcov: version magic: 0x41383552 [ 0.129390] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.130197] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.131348] pinctrl core: initialized pinctrl subsystem [ 0.132308] [ 0.133012] ************************************************************* [ 0.134014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.135013] ** ** [ 0.136013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.137013] ** ** [ 0.138015] ** This means that this kernel is built to expose internal ** [ 0.139013] ** IOMMU data structures, which may compromise security on ** [ 0.140015] ** your system. ** [ 0.141013] ** ** [ 0.142012] ** If you see this message and you are not debugging the ** [ 0.143013] ** kernel, report this immediately to your vendor! ** [ 0.144013] ** ** [ 0.145012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146014] ************************************************************* [ 0.147980] NET: Registered protocol family 16 [ 0.148659] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.149063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.150060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.152148] cpuidle: using governor menu [ 0.154511] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.158869] PCI: Using configuration type 1 for base access [ 0.162139] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.171418] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.172072] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.174026] cryptd: max_cpu_qlen set to 1000 [ 0.176284] ACPI: Added _OSI(Module Device) [ 0.177013] ACPI: Added _OSI(Processor Device) [ 0.178015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.179011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.184463] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.187588] ACPI: Interpreter enabled [ 0.188068] ACPI: PM: (supports S0 S3 S4 S5) [ 0.189013] ACPI: Using IOAPIC for interrupt routing [ 0.190164] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.191411] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.203628] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.204054] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.205028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.206096] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.209280] acpiphp: Slot [2] registered [ 0.210281] acpiphp: Slot [5] registered [ 0.211239] acpiphp: Slot [6] registered [ 0.212184] acpiphp: Slot [7] registered [ 0.213466] acpiphp: Slot [8] registered [ 0.214174] acpiphp: Slot [9] registered [ 0.215203] acpiphp: Slot [10] registered [ 0.216141] acpiphp: Slot [3] registered [ 0.217124] acpiphp: Slot [4] registered [ 0.218130] acpiphp: Slot [11] registered [ 0.219093] acpiphp: Slot [12] registered [ 0.220104] acpiphp: Slot [13] registered [ 0.221106] acpiphp: Slot [14] registered [ 0.222088] acpiphp: Slot [15] registered [ 0.223160] acpiphp: Slot [16] registered [ 0.224112] acpiphp: Slot [17] registered [ 0.225110] acpiphp: Slot [18] registered [ 0.226316] acpiphp: Slot [19] registered [ 0.227101] acpiphp: Slot [20] registered [ 0.228086] acpiphp: Slot [21] registered [ 0.229156] acpiphp: Slot [22] registered [ 0.230121] acpiphp: Slot [23] registered [ 0.231119] acpiphp: Slot [24] registered [ 0.232085] acpiphp: Slot [25] registered [ 0.233092] acpiphp: Slot [26] registered [ 0.234209] acpiphp: Slot [27] registered [ 0.235106] acpiphp: Slot [28] registered [ 0.236179] acpiphp: Slot [29] registered [ 0.237112] acpiphp: Slot [30] registered [ 0.238123] acpiphp: Slot [31] registered [ 0.239075] PCI host bridge to bus 0000:00 [ 0.240021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.241022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.242020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.243023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.244022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.245028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.246261] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.248330] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.251296] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.259031] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.263067] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.264111] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.265337] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.266137] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.269166] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.271198] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.275059] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.282000] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.295022] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.315019] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.323022] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.339279] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.361030] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.377023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.421026] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.454287] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.471028] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.503092] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.557025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.600017] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.643023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.672022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.740031] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.760000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.778055] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.822039] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.855026] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.881675] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.888027] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.907621] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.946023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.966045] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.977023] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.992034] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.036026] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.075000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.078827] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.083354] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.089603] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.092547] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.102128] iommu: Default domain type: Passthrough [ 1.103425] SCSI subsystem initialized [ 1.104166] ACPI: bus type USB registered [ 1.105523] usbcore: registered new interface driver usbfs [ 1.106132] usbcore: registered new interface driver hub [ 1.107123] usbcore: registered new device driver usb [ 1.115202] pps_core: LinuxPPS API ver. 1 registered [ 1.123015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.136062] PTP clock support registered [ 1.143147] EDAC MC: Ver: 3.0.0 [ 1.146163] PCI: Using ACPI for IRQ routing [ 1.153517] NetLabel: Initializing [ 1.158016] NetLabel: domain hash size = 128 [ 1.165014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.171088] NetLabel: unlabeled traffic allowed by default [ 1.178235] vgaarb: loaded [ 1.184101] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.185022] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.192180] clocksource: Switched to clocksource kvm-clock [ 1.367514] VFS: Disk quotas dquot_6.6.0 [ 1.369676] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.387504] *** VALIDATE ramfs *** [ 1.390617] *** VALIDATE hugetlbfs *** [ 1.395526] pnp: PnP ACPI init [ 1.400826] pnp: PnP ACPI: found 6 devices [ 1.425021] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.431168] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.434505] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.438165] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.444674] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.451019] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.459923] NET: Registered protocol family 2 [ 1.466347] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.479516] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.484596] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.493506] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.498542] TCP: Hash tables configured (established 65536 bind 65536) [ 1.504042] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.507464] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.510245] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.513856] NET: Registered protocol family 1 [ 1.516094] RPC: Registered named UNIX socket transport module. [ 1.518558] RPC: Registered udp transport module. [ 1.520238] RPC: Registered tcp transport module. [ 1.521757] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.523847] NET: Registered protocol family 44 [ 1.525685] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.527368] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.529109] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.531557] PCI: CLS 0 bytes, default 64 [ 1.533661] Unpacking initramfs... [ 3.839306] debug: unmapping init [mem 0xffff91d37cc54000-0xffff91d37ffbffff] [ 3.869238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.874481] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.890284] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 5.761977] Initialise system trusted keyrings [ 5.765950] Key type blacklist registered [ 5.774048] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.794489] zbud: loaded [ 5.803444] *** VALIDATE nfs *** [ 5.806782] *** VALIDATE nfs4 *** [ 5.809671] pstore: using deflate compression [ 5.816583] Platform Keyring initialized [ 6.001961] NET: Registered protocol family 38 [ 6.003945] Key type asymmetric registered [ 6.008414] Asymmetric key parser 'x509' registered [ 6.010300] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 6.013606] io scheduler mq-deadline registered [ 6.015344] io scheduler kyber registered [ 6.018358] io scheduler bfq registered [ 6.020445] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 6.024143] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 6.027990] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.032303] ACPI: Power Button [PWRF] [ 6.044325] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 6.054162] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 6.089387] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 6.114493] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 6.195647] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.245729] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 6.315612] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.329691] Non-volatile memory driver v1.3 [ 6.334825] Linux agpgart interface v0.103 [ 6.401974] virtio_blk virtio1: [vda] 145920 512-byte logical blocks (74.7 MB/71.3 MiB) [ 6.406359] vda: detected capacity change from 0 to 74711040 [ 6.428808] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 6.433310] vdb: detected capacity change from 0 to 1073741824 [ 6.454478] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.459100] vdc: detected capacity change from 0 to 2621440000 [ 6.481371] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.484894] vdd: detected capacity change from 0 to 2621440000 [ 6.505342] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.509576] vde: detected capacity change from 0 to 4294967296 [ 6.533627] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.538273] vdf: detected capacity change from 0 to 4294967296 [ 6.548128] libphy: Fixed MDIO Bus: probed [ 6.554202] usbcore: registered new interface driver usbserial_generic [ 6.557627] usbserial: USB Serial support registered for generic [ 6.560156] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.570175] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.576486] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.585149] mousedev: PS/2 mouse device common for all mice [ 6.593820] rtc_cmos 00:05: RTC can wake from S4 [ 6.600115] rtc_cmos 00:05: registered as rtc0 [ 6.604670] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.610486] intel_pstate: CPU model not supported [ 6.619873] hid: raw HID events driver (C) Jiri Kosina [ 6.620954] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.630944] usbcore: registered new interface driver usbhid [ 6.630951] usbhid: USB HID core driver [ 6.631276] drop_monitor: Initializing network drop monitor service [ 6.631470] Initializing XFRM netlink socket [ 6.631840] NET: Registered protocol family 10 [ 6.639165] Segment Routing with IPv6 [ 6.664660] NET: Registered protocol family 17 [ 6.669062] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.674718] mpls_gso: MPLS GSO support [ 6.691756] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.698235] RAS: Correctable Errors collector initialized. [ 6.700333] AVX version of gcm_enc/dec engaged. [ 6.702182] AES CTR mode by8 optimization enabled [ 6.830924] sched_clock: Marking stable (6830894777, 0)->(8238636773, -1407741996) [ 6.834131] registered taskstats version 1 [ 6.836954] Loading compiled-in X.509 certificates [ 6.842975] zswap: loaded using pool lzo/zbud [ 6.893741] Key type big_key registered [ 6.917322] Key type encrypted registered [ 6.918665] ima: No TPM chip found, activating TPM-bypass! [ 6.920770] ima: Allocated hash algorithm: sha1 [ 6.922301] ima: No architecture policies found [ 6.924927] evm: Initialising EVM extended attributes: [ 6.926567] evm: security.selinux [ 6.928215] evm: security.ima [ 6.929562] evm: security.capability [ 6.930664] evm: HMAC attrs: 0x1 [ 6.932937] rtc_cmos 00:05: setting system clock to 2026-08-17 06:09:28 UTC (1786946968) [ 6.940238] debug: unmapping init [mem 0xffffffffa8a03000-0xffffffffa8bfffff] [ 6.942896] debug: unmapping init [mem 0xffffffffa7782000-0xffffffffa7a58fff] [ 6.951365] Write protecting the kernel read-only data: 28672k [ 6.958579] debug: unmapping init [mem 0xffffffffa5e03000-0xffffffffa5ffffff] [ 6.962918] debug: unmapping init [mem 0xffffffffa6714000-0xffffffffa67fffff] [ 7.024307] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.036424] systemd[1]: Detected virtualization kvm. [ 7.039485] systemd[1]: Detected architecture x86-64. [ 7.041366] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.074012] systemd[1]: No hostname configured. [ 7.076195] systemd[1]: Set hostname to . [ 7.078788] random: systemd: uninitialized urandom read (16 bytes read) [ 7.082197] systemd[1]: Initializing machine ID from random generator. [ 7.190654] random: ln: uninitialized urandom read (6 bytes read) [ 7.429551] random: systemd: uninitialized urandom read (16 bytes read) [ 7.437813] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 7.455693] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 7.483737] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 10.403523] device-mapper: uevent: version 1.0.3 [ 10.407115] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 13.066797] virtio_net virtio0 ens2: renamed from eth0 [ 13.277538] random: fast init done [ 13.578252] scsi host0: ata_piix [ 13.614904] scsi host1: ata_piix [ 13.616336] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 13.620276] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 19.459652] random: crng init done [ 19.472580] random: 7 urandom warning(s) missed due to ratelimiting [ 22.863640] dracut-initqueue[584]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 26.070470] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 30.134224] printk: systemd: 26 output lines suppressed due to ratelimiting [ 30.957781] SELinux: Disabled at runtime. [ 31.120367] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 31.164254] systemd[1]: Detected virtualization kvm. [ 31.173781] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 34.711627] systemd[1]: initrd-switch-root.service: Succeeded. [ 34.728725] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 34.762677] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 34.782445] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 34.798293] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 34.838162] systemd[1]: Starting Journal Service... Starting Journal Service... [ 34.853313] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ 36.183760] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. [ 36.294849] systemd[1]: Starting Apply Kernel Variables... Starting Apply Kernel Variables...[ 36.324796] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 36.369462] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ 36.436185] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 36.589429] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ 36.665716] systemd[1]: sysroot-var-lib-nfs-rpc_pipefs.mount: Succeeded. [ 36.674466] systemd[1]: sysroot.mount: Succeeded. [ 36.693182] systemd[1]: Started Journal Service. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 38.618450] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 40.833049] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 41.322970] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 42.777484] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 43.238077] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…only root support (10s / no limit) [** ] A start job is running for Configur…only root support (11s / no limit) [*** ] A start job is running for Configur…only root support (11s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ ***] A start job is running for Configur…only root support (13s / no limit) [ **] A start job is running for Configur…only root support (13s / no limit) [ *] A start job is running for Configur…only root support (14s / no limit) [ **] A start job is running for Configur…only root support (14s / no limit) [ ***] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (16s / no limit) [*** ] A start job is running for Configur…only root support (16s / no limit) [** ] A start job is running for Configur…only root support (17s / no limit) [* ] A start job is running for Configur…only root support (17s / no limit) [** ] A start job is running for Configur…only root support (18s / no limit) [*** ] A start job is running for Configur…only root support (18s / no limit) [ *** ] A start job is running for Configur…only root support (19s / no limit) [ *** ] A start job is running for Configur…only root support (19s / no limit)[ 54.448147] Key type dns_resolver registered [ ***] A start job is running for Configur…only root support (20s / no limit) [ **] A start job is running for Configur…only root support (20s / no limit) [ *] A start job is running for Configur…only root support (21s / no limit) [ **] A start job is running for Configur…only root support (21s / no limit)[ 56.194448] NFS: Registering the id_resolver key type [ 56.216593] Key type id_resolver registered [ 56.221373] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (22s / no limit) [ *** ] A start job is running for Configur…only root support (22s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg418-server login: [ 76.857321] hrtimer: interrupt took 3836855 ns [ 150.636975] libcfs: loading out-of-tree module taints kernel. [ 150.703342] Key type ._llcrypt registered [ 150.708177] Key type .llcrypt registered [ 150.915961] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 172.402287] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 173.852314] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 173.871020] alg: No test for adler32 (adler32-zlib) [ 175.351247] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 176.288574] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 178.056222] Key type lgssc registered [ 180.050359] Lustre: Echo OBD driver; http://www.lustre.org/ [ 200.910097] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 250.159602] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 265.554807] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 265.659563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 266.978945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 267.045108] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 267.170773] Lustre: lustre-MDT0000: new disk, initializing [ 267.335394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 267.365201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 273.223549] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 290.266790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 290.403947] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 290.488238] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 290.503628] Lustre: Skipped 1 previous similar message [ 290.584208] Lustre: lustre-MDT0001: new disk, initializing [ 290.724489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 290.787905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 290.799622] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 295.460934] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 300.277512] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 310.124319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 310.314335] Lustre: lustre-OST0000: new disk, initializing [ 310.318456] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 310.324836] Lustre: 8454:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 310.386454] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 314.963716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 314.994105] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 315.113236] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 319.092599] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 339.233325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 339.336467] Lustre: lustre-OST0001: new disk, initializing [ 339.339729] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 339.344737] Lustre: 9529:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 339.419419] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 347.742398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 347.753504] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 347.911539] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 348.261418] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 362.057947] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 369.682577] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 378.124828] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing check_logdir /tmp/testlogs/ [ 385.184213] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing yml_node [ 391.764082] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 395.155204] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 398.648620] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 400.817930] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 17 02:16:00 EDT 2026 [ 423.791677] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 425.887989] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 445.414207] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 445.416901] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 445.451910] Lustre: Skipped 3 previous similar messages [ 445.484335] Lustre: Skipped 3 previous similar messages [ 450.534519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 450.550280] Lustre: Skipped 3 previous similar messages [ 451.559499] Lustre: server umount lustre-MDT0000 complete [ 457.324348] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786947418 with bad export cookie 9287146564269715106 [ 457.352085] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 457.361827] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 460.769284] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 460.772526] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 460.772843] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 460.796794] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 460.842922] Lustre: Skipped 1 previous similar message [ 462.817698] LustreError: 6526:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 462.818088] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 462.880446] Lustre: Skipped 2 previous similar messages [ 464.062236] Lustre: server umount lustre-MDT0001 complete [ 470.436169] Lustre: server umount lustre-OST0000 complete [ 475.591564] Lustre: server umount lustre-OST0001 complete [ 484.968288] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 501.779652] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 553.794910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 554.150864] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 554.217035] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 554.369001] Lustre: lustre-MDT0000: new disk, initializing [ 554.528608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 554.570292] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 561.006311] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 577.379073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 577.563295] Lustre: 15114:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 577.640651] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 577.648548] Lustre: Skipped 1 previous similar message [ 577.762956] Lustre: lustre-MDT0001: new disk, initializing [ 577.872306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 577.928471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 577.949067] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 583.768603] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 596.779881] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 604.216214] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 613.876317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 614.234756] Lustre: lustre-OST0000: new disk, initializing [ 614.250100] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 614.270060] Lustre: 17084:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 614.376058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 615.774632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 615.800532] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 615.978675] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 622.961918] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 635.379067] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 642.488350] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 642.777915] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 649.301694] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 649.520613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 656.877177] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 656.889668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 656.898679] Lustre: Skipped 1 previous similar message [ 656.921018] Lustre: Skipped 1 previous similar message [ 658.709937] Lustre: server umount lustre-OST0000 complete [ 665.059313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 665.076436] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 665.112430] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 669.934993] Lustre: server umount lustre-MDT0000 complete [ 672.225382] LustreError: 15125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 672.263343] LustreError: 15125:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 674.839471] LustreError: 15105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786947636 with bad export cookie 9287146564269716632 [ 674.843266] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 674.866715] LustreError: 15105:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 675.282883] Lustre: server umount lustre-MDT0001 complete [ 682.176399] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 02:20:41 (1786947641) [ 691.297278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 692.172682] LustreError: 19352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 692.356577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 697.832831] LustreError: 19353:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 698.123945] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 707.054740] LustreError: 19352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 707.081595] LustreError: 19352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 708.986808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 715.360223] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 726.785654] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 734.946333] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 743.076656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 750.299548] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 759.742336] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 769.675030] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 777.196267] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 777.203309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 777.215828] Lustre: Skipped 3 previous similar messages [ 777.219127] Lustre: Skipped 3 previous similar messages [ 783.086922] Lustre: server umount lustre-MDT0000 complete [ 787.209137] LustreError: 19333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786947748 with bad export cookie 9287146564269717521 [ 787.230677] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 787.235281] LustreError: 19333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 787.435244] LustreError: 20793:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 787.459540] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 787.570976] Lustre: server umount lustre-MDT0001 complete [ 794.628294] Lustre: server umount lustre-OST0000 complete [ 821.078528] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 835.607713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 836.153341] LustreError: 23737:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 836.200874] LustreError: 23737:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 836.264948] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 836.272936] Lustre: Skipped 2 previous similar messages [ 841.554678] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 852.287490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 858.936609] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 862.741397] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 870.917432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 879.738525] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 891.840885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 892.069682] Lustre: lustre-OST0001: new disk, initializing [ 892.074755] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 892.088884] Lustre: 26053:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 894.094786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 894.108400] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 894.368585] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 900.610183] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 912.182323] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 916.196663] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 934.891843] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 934.903038] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 934.915329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 934.921893] Lustre: Skipped 4 previous similar messages [ 938.575958] Lustre: server umount lustre-MDT0000 complete [ 940.534129] LustreError: 23737:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 940.559454] LustreError: 23737:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 943.545573] LustreError: 23720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786947905 with bad export cookie 9287146564269718368 [ 943.555991] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 943.956355] Lustre: server umount lustre-MDT0001 complete [ 959.450787] Lustre: server umount lustre-OST0000 complete [ 961.507186] Lustre: 3651:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786947907/real 1786947907] req@ffff91d3f1734a80 x1873749881397248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786947923 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 963.975534] Lustre: server umount lustre-OST0001 complete [ 985.248370] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 998.151658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 998.647875] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 998.658671] Lustre: Skipped 3 previous similar messages [ 1004.143179] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1013.770441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1019.522421] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1022.987128] Lustre: 30223:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1030.587375] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1038.292820] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1048.662625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1055.853780] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1066.114942] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1070.602353] Lustre: 32101:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1079.782080] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1079.791793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1079.792117] Lustre: Skipped 6 previous similar messages [ 1079.798148] Lustre: Skipped 5 previous similar messages [ 1084.353366] Lustre: server umount lustre-MDT0000 complete [ 1084.915511] LustreError: 32219:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1084.942981] LustreError: 32219:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 1088.753902] LustreError: 29056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786948050 with bad export cookie 9287146564269719663 [ 1088.756221] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1088.767245] LustreError: 29056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1089.117525] Lustre: server umount lustre-MDT0001 complete [ 1104.402503] Lustre: server umount lustre-OST0000 complete [ 1106.144103] Lustre: 3651:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786948051/real 1786948051] req@ffff91d2c225a680 x1873749881448960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786948067 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1108.569745] Lustre: server umount lustre-OST0001 complete [ 1119.084552] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 02:27:58 (1786948078) [ 1152.784626] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 1163.680115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1168.756287] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1178.086516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.422980] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1187.032475] Lustre: 36058:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1194.563390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1202.453159] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1211.541514] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1218.345531] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1227.047309] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1231.353958] Lustre: 37935:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1237.477918] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1237.484638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1237.490220] Lustre: Skipped 4 previous similar messages [ 1237.498200] Lustre: Skipped 5 previous similar messages [ 1243.302402] Lustre: server umount lustre-MDT0000 complete [ 1248.191869] LustreError: 37937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786948209 with bad export cookie 9287146564269720769 [ 1248.207050] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1248.218850] LustreError: 37937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1248.660834] Lustre: server umount lustre-MDT0001 complete [ 1262.817556] Lustre: server umount lustre-OST0000 complete [ 1278.120874] Lustre: server umount lustre-OST0001 complete [ 1289.226808] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 02:30:48 (1786948248) [ 1309.764152] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 1321.176152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1321.710228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1321.723604] Lustre: Skipped 7 previous similar messages [ 1327.010268] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1336.802179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1343.069089] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1346.298356] Lustre: 41230:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1354.186959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1361.765392] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1361.843932] LustreError: 41586:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1361.870542] LustreError: 41586:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 1372.679231] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1381.952083] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1392.334868] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1397.142165] Lustre: 43108:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1398.321057] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1398.329667] Lustre: Skipped 1 previous similar message [ 1403.888498] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1403.910799] Lustre: Skipped 2 previous similar messages [ 1403.920087] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1404.289962] Lustre: server umount lustre-MDT0000 complete [ 1408.771803] LustreError: 40059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786948370 with bad export cookie 9287146564269721875 [ 1408.781908] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1408.791396] LustreError: 40059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1423.328912] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1423.601448] Lustre: server umount lustre-MDT0001 complete [ 1432.775549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1433.570131] LustreError: 3648:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff91d3ed9d3480 x1873749881557120/t0(0) o250->MGC192.168.204.118@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1439.106697] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1447.902499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1449.522784] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1453.361848] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1463.469798] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1465.558585] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1473.697146] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1476.227351] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1484.261391] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1487.969030] Lustre: server umount lustre-MDT0000 complete [ 1492.473897] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786948454 with bad export cookie 9287146564269722967 [ 1492.476822] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1492.505637] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1492.870406] Lustre: server umount lustre-MDT0001 complete [ 1508.030517] Lustre: server umount lustre-OST0000 complete [ 1511.968213] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786948457/real 1786948457] req@ffff91d400f45c00 x1873749881595264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786948473 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1512.982497] Lustre: server umount lustre-OST0001 complete [ 1523.016919] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 02:34:42 (1786948482) [ 1541.582977] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 1551.536033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1556.607881] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1565.773298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1570.631906] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1573.882528] Lustre: 48872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1581.272172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1589.361671] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1598.161690] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1605.875316] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1615.784633] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1621.562224] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1635.301589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1635.310681] Lustre: Skipped 24 previous similar messages [ 1640.402681] Lustre: server umount lustre-MDT0000 complete [ 1644.909734] LustreError: 47698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786948606 with bad export cookie 9287146564269723793 [ 1644.920016] LustreError: 47698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1644.920720] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1645.382426] Lustre: server umount lustre-MDT0001 complete [ 1659.960918] Lustre: server umount lustre-OST0000 complete [ 1661.728460] Lustre: 3652:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786948607/real 1786948607] req@ffff91d2c34b4700 x1873749881653888/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786948623 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1661.762607] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1661.773329] Lustre: Skipped 13 previous similar messages [ 1663.822395] Lustre: server umount lustre-OST0001 complete [ 1675.513201] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 02:37:14 (1786948634) [ 1677.921590] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1679.605621] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 02:37:19 (1786948639) [ 1692.452207] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 1705.905449] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 1734.339895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1734.580378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1734.671374] Lustre: lustre-MDT0000: new disk, initializing [ 1734.800750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1740.698743] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1752.322884] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1760.430977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1760.782902] Lustre: lustre-OST0000: new disk, initializing [ 1760.786948] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1760.794571] Lustre: Skipped 1 previous similar message [ 1760.801548] Lustre: 54955:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1762.603886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1762.615856] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1762.676864] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1767.052021] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1777.063600] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1779.560955] Lustre: DEBUG MARKER: create test files [ 1819.123136] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1824.526888] Lustre: server umount lustre-OST0000 complete [ 1829.488392] Lustre: server umount lustre-MDT0000 complete [ 1843.850446] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 1846.781702] Key type lgssc unregistered [ 1847.122738] LNet: 56589:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1847.128226] LNetError: 56589:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1847.154676] LNet: Removed LNI 192.168.204.118@tcp [ 1848.249171] Key type .llcrypt unregistered [ 1848.251069] Key type ._llcrypt unregistered [ 1879.892849] Key type ._llcrypt registered [ 1879.896417] Key type .llcrypt registered [ 1880.038100] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 1881.794361] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1881.831958] alg: No test for adler32 (adler32-zlib) [ 1882.974125] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 1883.316876] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 1885.024275] Key type lgssc registered [ 1886.447604] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1894.701042] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1894.735200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1896.245624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1901.255231] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1909.749140] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1916.722962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1917.051162] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1921.138658] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1921.312445] Lustre: 59325:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1921.328367] Lustre: 59325:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1921.337245] Lustre: 59325:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1921.350592] Lustre: 59325:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1921.370621] Lustre: 59325:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1921.387226] Lustre: 59325:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1924.093166] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 1932.148376] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1937.914692] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1944.594124] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1957.346339] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1957.374236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1957.376723] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1960.502975] Lustre: server umount lustre-OST0000 complete [ 1966.334172] Lustre: server umount lustre-MDT0000 complete [ 1979.006536] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 1981.804296] Key type lgssc unregistered [ 1982.053238] LNet: 60939:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1982.066934] LNetError: 60939:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1983.144428] LNet: Removed LNI 192.168.204.118@tcp [ 1984.381654] Key type .llcrypt unregistered [ 1984.385394] Key type ._llcrypt unregistered [ 2024.181787] Key type ._llcrypt registered [ 2024.184603] Key type .llcrypt registered [ 2024.319210] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 2025.473546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2025.497550] alg: No test for adler32 (adler32-zlib) [ 2026.756719] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 2027.064113] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 2028.801724] Key type lgssc registered [ 2030.549958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2043.567550] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2043.579601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2045.040539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2050.309835] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2058.439619] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2066.666756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2067.032758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2070.136115] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2070.228751] Lustre: 63781:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2070.236889] Lustre: 63781:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2070.244829] Lustre: 63781:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2070.249266] Lustre: 63781:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2070.256779] Lustre: 63781:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2070.261545] Lustre: 63781:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2073.570887] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2081.801468] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2087.206341] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2101.218198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2101.220448] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2101.230420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2103.460798] Lustre: server umount lustre-OST0000 complete [ 2108.120944] Lustre: server umount lustre-MDT0000 complete [ 2121.112896] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 2124.347364] Key type lgssc unregistered [ 2124.776904] LNet: 65341:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2124.789936] LNetError: 65341:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2124.829132] LNet: Removed LNI 192.168.204.118@tcp [ 2126.009195] Key type .llcrypt unregistered [ 2126.014071] Key type ._llcrypt unregistered [ 2146.815844] Key type ._llcrypt registered [ 2146.822776] Key type .llcrypt registered [ 2146.926839] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 2164.870884] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 2166.193307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2166.325749] alg: No test for adler32 (adler32-zlib) [ 2167.321528] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 2167.657790] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 2169.408152] Key type lgssc registered [ 2170.906884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2214.862219] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2214.904865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2216.212400] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2216.243611] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2216.334729] Lustre: lustre-MDT0000: new disk, initializing [ 2216.421683] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2216.443939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2221.533589] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2233.891390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2234.011943] Lustre: 68500:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2234.064216] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2234.067822] Lustre: Skipped 1 previous similar message [ 2234.138818] Lustre: lustre-MDT0001: new disk, initializing [ 2234.265757] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2234.316063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2234.326672] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2239.912396] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2251.076930] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2257.808165] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2267.265540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2267.581146] Lustre: lustre-OST0000: new disk, initializing [ 2267.584806] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2267.591927] Lustre: 70469:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2267.668355] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2274.417851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2274.427753] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2274.511326] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2274.611126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2285.463610] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2290.555270] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2290.771512] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2295.948681] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2296.170948] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2300.386744] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2300.391683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2300.396077] LustreError: Skipped 1 previous similar message [ 2300.397329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2300.418915] Lustre: Skipped 1 previous similar message [ 2304.734800] Lustre: server umount lustre-OST0000 complete [ 2310.632338] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2310.645056] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2310.653763] Lustre: Skipped 1 previous similar message [ 2310.674079] Lustre: Skipped 2 previous similar messages [ 2315.117820] Lustre: server umount lustre-MDT0000 complete [ 2319.444687] LustreError: 69294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786949281 with bad export cookie 10395059569355530637 [ 2319.453644] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2319.458933] LustreError: 69294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2319.884618] Lustre: server umount lustre-MDT0001 complete [ 2331.085693] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:48:10 (1786949290) [ 2344.742702] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 2390.421467] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 2402.730314] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 2444.717868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2445.085688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2445.132624] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2445.185925] Lustre: lustre-MDT0000: new disk, initializing [ 2445.244398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2445.256052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2450.340903] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2463.133690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2463.333479] Lustre: 76492:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2463.388392] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2463.391898] Lustre: Skipped 1 previous similar message [ 2463.494359] Lustre: lustre-MDT0001: new disk, initializing [ 2463.675891] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2463.737335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2463.756336] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2468.911691] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2480.314438] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2487.018889] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2494.957597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2495.336933] Lustre: lustre-OST0000: new disk, initializing [ 2495.346714] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2495.359524] Lustre: 78461:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2495.469682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2502.015701] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2502.238239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2502.257552] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2502.434129] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2513.239880] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2518.157828] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2518.397990] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2523.861243] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2524.024524] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2528.226873] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2528.229596] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2528.238353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2528.250843] Lustre: Skipped 1 previous similar message [ 2532.294313] Lustre: server umount lustre-OST0000 complete [ 2538.466355] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2538.480158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2538.485457] Lustre: Skipped 2 previous similar messages [ 2542.815582] Lustre: server umount lustre-MDT0000 complete [ 2543.585255] LustreError: 76502:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2543.600684] LustreError: 76502:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2546.869235] LustreError: 76482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786949508 with bad export cookie 10395059569355531519 [ 2546.875275] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2546.886777] LustreError: 76482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2547.574229] Lustre: server umount lustre-MDT0001 complete [ 2556.049683] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:51:55 (1786949515) [ 2562.653846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2563.151983] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2563.307270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2568.354650] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2568.686740] LustreError: 80872:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2572.843189] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2577.811972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2577.895341] LustreError: 80872:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2582.845734] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2590.184144] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2597.131333] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2604.485138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2604.804104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2604.809496] Lustre: Skipped 1 previous similar message [ 2611.620530] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2620.063857] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2624.307932] Lustre: 83542:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting a7dae31b-dcca-48f7-84a9-7207505b85de at adminstrative request [ 2628.584080] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2628.610922] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2638.822328] Lustre: MGS: Client a7dae31b-dcca-48f7-84a9-7207505b85de (at 0@lo) reconnecting [ 2638.835653] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x9042ab9a5a762871 to 0x9042ab9a5a762b11 [ 2638.849602] Lustre: MGC192.168.204.118@tcp: Connection restored to 0@lo (at 0@lo) [ 2662.825265] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2663.116506] Lustre: lustre-OST0001: new disk, initializing [ 2663.126497] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2663.140974] Lustre: 83934:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2670.379246] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2671.754490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2671.776664] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2671.899476] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2680.933460] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2687.465055] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2687.499469] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2687.514854] Lustre: Skipped 1 previous similar message [ 2687.522480] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2687.530121] Lustre: Skipped 1 previous similar message [ 2690.572455] Lustre: server umount lustre-OST0001 complete [ 2692.593408] LustreError: 82693:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2692.617697] LustreError: 82693:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2695.146164] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2695.153713] LustreError: Skipped 2 previous similar messages [ 2697.704743] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2697.714242] Lustre: Skipped 3 previous similar messages [ 2701.156537] Lustre: server umount lustre-OST0000 complete [ 2706.407150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2706.420143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2706.439110] Lustre: Skipped 3 previous similar messages [ 2718.177666] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2718.188829] Lustre: Skipped 6 previous similar messages [ 2719.200298] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2719.548323] Lustre: server umount lustre-MDT0000 complete [ 2723.302589] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2723.331602] LustreError: 80871:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2724.850820] LustreError: 80852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786949686 with bad export cookie 10395059569355533073 [ 2724.859767] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2724.885527] LustreError: 80852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2725.269825] Lustre: server umount lustre-MDT0001 complete [ 2741.076397] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:54:59 (1786949699) [ 2745.080775] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2747.998225] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:55:06 (1786949706) [ 2750.593796] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2752.679678] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:55:12 (1786949712) [ 2754.299597] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2756.986899] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:55:15 (1786949715) [ 2759.512243] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2763.016872] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:55:21 (1786949721) [ 2765.707650] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2772.775240] loop: module loaded [ 2772.796064] loop0: detected capacity change from 0 to 4294967296 [ 2772.858489] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2774.660467] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2779.109664] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2784.232427] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2814.091432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2814.156895] LustreError: 88065:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2814.162509] LustreError: 88065:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2814.170359] LustreError: 88065:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2814.174229] LustreError: 88065:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2838.765982] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 2849.498443] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 2888.550129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2888.786868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2888.811761] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2888.879636] Lustre: lustre-MDT0000: new disk, initializing [ 2888.964979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2888.974847] Lustre: Skipped 1 previous similar message [ 2888.989952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2894.710267] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2906.484652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2906.650311] Lustre: 90930:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2906.879406] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2911.868565] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2921.339629] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2927.964469] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2934.940795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2935.180886] Lustre: 92901:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2942.140884] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 2944.110944] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2944.255621] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2951.559351] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2956.369503] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2956.607101] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2961.649899] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2961.892891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2964.963334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2964.964030] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2964.966124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2964.975291] LustreError: Skipped 1 previous similar message [ 2965.033418] Lustre: Skipped 2 previous similar messages [ 2970.350970] Lustre: server umount lustre-OST0000 complete [ 2980.726259] Lustre: server umount lustre-MDT0000 complete [ 2984.499828] LustreError: 90924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786949946 with bad export cookie 10395059569355533689 [ 2984.501833] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2984.513365] LustreError: 90924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2984.830854] Lustre: server umount lustre-MDT0001 complete [ 2994.235623] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:59:13 (1786949953) [ 3001.759140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3002.224072] LustreError: 95313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3007.350708] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3016.679143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3016.968666] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 3016.975471] Lustre: Skipped 3 previous similar messages [ 3022.065356] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3031.078579] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3039.554562] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3048.192147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3055.270416] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3062.206719] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3067.314766] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3074.003644] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec [ 3079.153277] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3079.408168] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3087.084750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3087.283356] Lustre: lustre-OST0001: new disk, initializing [ 3087.291862] Lustre: Skipped 2 previous similar messages [ 3087.305903] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3087.310456] Lustre: Skipped 3 previous similar messages [ 3087.319948] Lustre: 98725:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3093.312328] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3093.698526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3093.707505] Lustre: Skipped 2 previous similar messages [ 3093.712452] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3093.789975] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3105.634886] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3111.593453] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3111.911116] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3118.116452] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3118.352976] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3139.734925] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3171.793169] Lustre: Failing over lustre-MDT0000 [ 3171.881214] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.18@tcp (stopping) [ 3171.891828] Lustre: Skipped 7 previous similar messages [ 3172.246814] Lustre: server umount lustre-MDT0000 complete [ 3175.908187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3175.909257] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3175.919281] LustreError: 95308:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3175.919292] LustreError: 95308:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3175.985427] Lustre: Skipped 4 previous similar messages [ 3192.301198] Lustre: 66720:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786950137/real 1786950137] req@00000000e0641b4e x1873751970171264/t0(0) o400->MGC192.168.204.118@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786950153 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3192.343552] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3198.467298] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3198.470092] LDISKFS-fs (dm-0): recovery complete [ 3198.481356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3201.570805] LustreError: 66718:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000c2157003 x1873751970179328/t0(0) o250->MGC192.168.204.118@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3201.982625] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3202.550348] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3202.559517] Lustre: 95309:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3207.154648] Lustre: 95314:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3207.161067] Lustre: 95314:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 3207.168664] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3207.217849] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3207.297292] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3207.627799] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3207.962603] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3207.983216] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3208.326164] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3208.346980] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3208.365475] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3208.373995] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 3209.017146] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3209.044110] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3209.377574] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3209.385329] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3210.668560] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3210.683192] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 3211.328533] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3211.338311] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 3211.656238] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3211.667378] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 3214.857893] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3214.874022] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 3215.539596] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3215.553123] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3215.872158] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3215.887305] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3223.161174] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3223.177108] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 3223.809032] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3223.818958] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 3224.136234] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3224.151948] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 3239.423400] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3239.447355] Lustre: 101595:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 3240.102458] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3240.107442] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3240.432325] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3240.449228] LustreError: 101595:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3262.120156] Lustre: 101595:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3262.147071] Lustre: 101595:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3262.164286] Lustre: 101595:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 8e0bf983-1cfe-4cb8-9445-9ab2b58ac057@192.168.204.18@tcp [ 3262.194357] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3262.203331] Lustre: 101595:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@0000000082eca024 x1873751951084160/t0(8589934768) o35->8e0bf983-1cfe-4cb8-9445-9ab2b58ac057@192.168.204.18@tcp:379/0 lens 392/0 e 0 to 0 dl 1786950234 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3262.263294] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3262.303061] Lustre: lustre-MDT0000: Denying connection for new client 8e0bf983-1cfe-4cb8-9445-9ab2b58ac057 (at 192.168.204.18@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3262.354250] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3262.394675] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3262.406642] Lustre: Skipped 2 previous similar messages [ 3262.425756] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3262.454512] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3262.455050] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3288.546187] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3288.549288] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3288.568170] LustreError: Skipped 1 previous similar message [ 3288.596299] Lustre: Skipped 2 previous similar messages [ 3291.859651] Lustre: server umount lustre-OST0000 complete [ 3302.525852] Lustre: server umount lustre-OST0001 complete [ 3308.518058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3308.529376] Lustre: Skipped 8 previous similar messages [ 3313.576724] Lustre: server umount lustre-MDT0000 complete [ 3314.659262] LustreError: 95313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3314.678510] LustreError: 95313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 3317.963625] LustreError: 95293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786950279 with bad export cookie 10395059569355584901 [ 3317.973535] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3317.982079] LustreError: 95293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3318.302611] Lustre: server umount lustre-MDT0001 complete [ 3328.373692] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:04:47 (1786950287) [ 3337.321637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3337.336103] Lustre: *** cfs_fail_loc=197, val=0*** [ 3352.992957] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3358.112493] LustreError: 103572:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.118@tcp: failed processing log, type 4: rc = -110 [ 3363.265184] Lustre: *** cfs_fail_loc=197, val=0*** [ 3363.272580] Lustre: Skipped 65 previous similar messages [ 3383.840659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3383.847376] Lustre: Skipped 3 previous similar messages [ 3391.460531] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3400.701204] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3403.083410] Lustre: server umount lustre-OST0000 complete [ 3413.659062] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:06:12 (1786950372) [ 3437.660903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3443.406514] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3454.780200] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3459.170418] LustreError: 105223:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3459.181514] LustreError: 105223:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3459.193685] LustreError: 105223:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3459.501161] Lustre: server umount lustre-MDT0000 complete [ 3498.053977] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 3508.327127] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 3547.764942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3548.106900] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3548.158668] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3548.286130] Lustre: lustre-MDT0000: new disk, initializing [ 3548.385830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3553.661837] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3565.531495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3565.739637] Lustre: 109400:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3566.053097] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3572.137280] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3586.006817] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3593.667303] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3602.975849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3603.214479] Lustre: 111371:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3612.510332] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3613.104280] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3624.887596] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3631.329469] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3631.726313] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3637.687457] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3638.105847] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3643.363311] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3643.369854] LustreError: Skipped 3 previous similar messages [ 3643.372533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3643.385115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3643.396411] Lustre: Skipped 5 previous similar messages [ 3643.401194] Lustre: Skipped 2 previous similar messages [ 3647.495677] Lustre: server umount lustre-OST0000 complete [ 3658.481986] Lustre: server umount lustre-MDT0000 complete [ 3658.725153] LustreError: 109411:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3658.757711] LustreError: 109411:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3663.363368] LustreError: 109392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786950624 with bad export cookie 10395059569355595478 [ 3663.367436] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3663.376625] LustreError: 109392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3663.878289] Lustre: server umount lustre-MDT0001 complete [ 3673.907855] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:10:33 (1786950633) [ 3684.144621] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 3687.178527] Key type lgssc unregistered [ 3687.486542] LNet: 113862:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3687.498336] LNetError: 113862:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3687.524967] LNet: Removed LNI 192.168.204.118@tcp [ 3688.431796] Key type .llcrypt unregistered [ 3688.433528] Key type ._llcrypt unregistered [ 3706.844436] Key type ._llcrypt registered [ 3706.848136] Key type .llcrypt registered [ 3706.986700] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 3723.019494] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 3724.216946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3724.238839] alg: No test for adler32 (adler32-zlib) [ 3725.510739] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 3725.977410] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 3727.672124] Key type lgssc registered [ 3728.966928] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3774.973620] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3774.993180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3776.247206] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3776.288682] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3776.385347] Lustre: lustre-MDT0000: new disk, initializing [ 3776.661398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3776.689732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3781.634107] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3794.254267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3794.592560] Lustre: lustre-OST0000: new disk, initializing [ 3794.599747] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3794.605136] Lustre: Skipped 1 previous similar message [ 3794.614074] Lustre: 118035:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3794.766072] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3801.348603] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3804.251394] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3804.277397] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3804.322710] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3813.642049] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3813.893174] Lustre: lustre-OST0001: new disk, initializing [ 3813.902223] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3813.912159] Lustre: 118898:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3814.062245] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3820.679564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3820.687397] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3820.717600] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3823.022881] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 3835.877280] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3835.929250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3840.570697] Lustre: server umount lustre-MDT0000 complete [ 3848.008331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3853.049505] LustreError: 117223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786950814 with bad export cookie 13687422884446731485 [ 3853.061173] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3853.068851] LustreError: 117223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3853.197779] Lustre: server umount lustre-OST0000 complete [ 3857.317289] Lustre: server umount lustre-OST0001 complete [ 3868.048062] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 03:13:47 (1786950827) [ 3897.883925] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 03:14:16 (1786950856) [ 3913.587875] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 3942.216176] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:15:01 (1786950901) [ 3987.799070] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 4000.876973] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 4042.709031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4043.015590] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4043.051778] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4043.130149] Lustre: lustre-MDT0000: new disk, initializing [ 4043.263981] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4043.283815] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4047.939664] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4060.685471] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4061.043797] Lustre: 126680:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4061.180681] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4061.189816] Lustre: Skipped 1 previous similar message [ 4061.340366] Lustre: lustre-MDT0001: new disk, initializing [ 4061.529020] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4061.573113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4061.594399] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4066.751595] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4081.092490] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4089.424406] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4099.769525] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4100.111469] Lustre: lustre-OST0000: new disk, initializing [ 4100.114566] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4100.119625] Lustre: 128651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4100.190477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4105.318981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4105.337345] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4105.516033] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4107.558058] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4120.206444] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4127.517891] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4127.830993] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4132.870512] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4133.113889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4136.417673] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4136.424974] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4136.443087] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4136.464022] Lustre: Skipped 2 previous similar messages [ 4136.467163] Lustre: Skipped 1 previous similar message [ 4141.542852] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4141.550840] Lustre: Skipped 2 previous similar messages [ 4142.620447] Lustre: server umount lustre-OST0000 complete [ 4148.717865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4148.737519] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.796913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4151.779814] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4154.005917] Lustre: server umount lustre-MDT0000 complete [ 4158.903450] LustreError: 126671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786951120 with bad export cookie 13687422884446732703 [ 4158.911062] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4158.914131] LustreError: 126671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4159.639375] Lustre: server umount lustre-MDT0001 complete [ 4170.882650] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:18:49 (1786951129) [ 4179.692857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4180.371658] LustreError: 131063:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4180.394444] LustreError: 131063:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4180.461709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4185.570804] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4185.653767] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4189.667043] LustreError: 131063:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4194.798873] LustreError: 131064:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4195.334040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4200.874478] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4212.862175] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4221.390162] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4228.368664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4228.859212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4228.867875] Lustre: Skipped 1 previous similar message [ 4235.263275] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4243.339524] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4258.265876] Lustre: *** cfs_fail_loc=159, val=0*** [ 4274.782390] Lustre: lustre-MDT0000: Client 58a05717-8772-4196-8624-cac133a9cd50 (at 192.168.204.18@tcp) reconnecting [ 4277.056479] Lustre: *** cfs_fail_loc=159, val=0*** [ 4277.067200] Lustre: Skipped 7 previous similar messages [ 4293.672580] Lustre: lustre-MDT0000: Client 58a05717-8772-4196-8624-cac133a9cd50 (at 192.168.204.18@tcp) reconnecting [ 4313.060869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4313.063038] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4313.063848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4313.063853] Lustre: Skipped 2 previous similar messages [ 4313.093238] Lustre: Skipped 1 previous similar message [ 4313.118521] LustreError: Skipped 1 previous similar message [ 4313.382447] Lustre: server umount lustre-OST0000 complete [ 4323.299181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4323.301617] Lustre: Skipped 4 previous similar messages [ 4323.518147] Lustre: server umount lustre-MDT0000 complete [ 4327.853646] LustreError: 131043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786951289 with bad export cookie 13687422884446733592 [ 4327.865325] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4327.873741] LustreError: 131043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4328.334922] Lustre: server umount lustre-MDT0001 complete [ 4339.688442] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 4342.388874] Key type lgssc unregistered [ 4342.708565] LNet: 134864:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4342.722481] LNetError: 134864:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4342.740687] LNet: Removed LNI 192.168.204.118@tcp [ 4343.924078] Key type .llcrypt unregistered [ 4343.926811] Key type ._llcrypt unregistered [ 4373.340801] Key type ._llcrypt registered [ 4373.345182] Key type .llcrypt registered [ 4373.470446] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 4374.709946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4374.736314] alg: No test for adler32 (adler32-zlib) [ 4375.770429] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 4375.976901] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 4377.657167] Key type lgssc registered [ 4378.876247] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4386.173218] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4386.193320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4387.690107] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4393.012166] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4398.062702] LustreError: 136283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4398.085651] LustreError: 136283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4403.173674] LustreError: 136284:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4404.314658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4404.757093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4409.588036] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4417.771794] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4424.383082] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4432.068422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4432.523312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4433.585781] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4438.877483] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4449.227381] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4461.857477] Lustre: *** cfs_fail_loc=159, val=0*** [ 4477.482243] Lustre: lustre-MDT0000: Client 022b1411-3068-42a5-8a53-583b86adccf1 (at 192.168.204.18@tcp) reconnecting [ 4480.038071] Lustre: *** cfs_fail_loc=159, val=0*** [ 4496.477498] Lustre: lustre-MDT0001: Client 022b1411-3068-42a5-8a53-583b86adccf1 (at 192.168.204.18@tcp) reconnecting [ 4498.442926] Lustre: *** cfs_fail_loc=159, val=0*** [ 4498.451098] Lustre: Skipped 3 previous similar messages [ 4514.897209] Lustre: lustre-MDT0001: Client 022b1411-3068-42a5-8a53-583b86adccf1 (at 192.168.204.18@tcp) reconnecting [ 4520.462354] Lustre: 139319:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4525.301183] Lustre: *** cfs_fail_loc=159, val=0*** [ 4525.315946] Lustre: Skipped 5 previous similar messages [ 4541.496741] Lustre: lustre-MDT0000: Client 2c94a8de-fe11-43dc-aaf0-78e93d5a706e (at 192.168.204.18@tcp) reconnecting [ 4543.605369] Lustre: *** cfs_fail_loc=159, val=0*** [ 4543.607549] Lustre: Skipped 13 previous similar messages [ 4559.372924] Lustre: lustre-MDT0000: Client 2c94a8de-fe11-43dc-aaf0-78e93d5a706e (at 192.168.204.18@tcp) reconnecting [ 4561.227663] Lustre: 139615:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4582.371117] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4582.388123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4582.409911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4583.411887] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4583.418552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4583.428506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4583.453751] Lustre: Skipped 1 previous similar message [ 4587.878774] Lustre: server umount lustre-OST0000 complete [ 4593.648348] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4593.653262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4593.669666] Lustre: Skipped 1 previous similar message [ 4593.680144] Lustre: Skipped 1 previous similar message [ 4598.755602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4598.767679] Lustre: Skipped 1 previous similar message [ 4599.411935] Lustre: server umount lustre-MDT0000 complete [ 4603.699406] LustreError: 139564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786951565 with bad export cookie 13286589766022816200 [ 4603.703616] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4603.712692] LustreError: 139564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4604.150106] Lustre: server umount lustre-MDT0001 complete [ 4617.625908] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 4621.271947] Key type lgssc unregistered [ 4621.750605] LNet: 140698:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4621.775576] LNetError: 140698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4621.808549] LNet: Removed LNI 192.168.204.118@tcp [ 4622.899648] Key type .llcrypt unregistered [ 4622.905398] Key type ._llcrypt unregistered [ 4650.015576] Key type ._llcrypt registered [ 4650.019919] Key type .llcrypt registered [ 4650.203646] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 4651.638943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4651.719662] alg: No test for adler32 (adler32-zlib) [ 4652.704400] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 4652.916219] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 4654.600281] Key type lgssc registered [ 4655.837644] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4663.973676] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4663.995147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4665.505099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4669.779800] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4676.079361] LustreError: 142114:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4676.130837] LustreError: 142114:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4679.787406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4680.675369] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4686.057554] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4697.156594] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4703.594893] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4713.320689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4713.899853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4720.724910] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4726.202612] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4730.174684] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4736.460068] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4746.692580] Lustre: 144716:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4750.480927] Lustre: 144765:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4752.590658] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4753.727074] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4758.987210] Lustre: 144923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4760.405863] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4761.766048] Lustre: 145021:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4767.215687] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4767.215760] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4767.231121] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4767.232085] LustreError: Skipped 1 previous similar message [ 4767.273737] Lustre: Skipped 1 previous similar message [ 4772.324149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4772.344019] Lustre: Skipped 1 previous similar message [ 4772.963246] Lustre: server umount lustre-OST0000 complete [ 4777.441491] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4777.447146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4777.454055] Lustre: Skipped 1 previous similar message [ 4777.467302] Lustre: Skipped 2 previous similar messages [ 4782.562504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4782.566519] Lustre: Skipped 1 previous similar message [ 4784.192626] Lustre: server umount lustre-MDT0000 complete [ 4787.682861] LustreError: 142841:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4787.731285] LustreError: 142841:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4789.537018] LustreError: 142094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786951751 with bad export cookie 6303944069354510212 [ 4789.543022] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4789.550656] LustreError: 142094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4790.240642] Lustre: server umount lustre-MDT0001 complete [ 4805.951896] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 4809.989828] Key type lgssc unregistered [ 4810.295330] LNet: 146082:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4810.301049] LNetError: 146082:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4810.349481] LNet: Removed LNI 192.168.204.118@tcp [ 4811.535389] Key type .llcrypt unregistered [ 4811.538150] Key type ._llcrypt unregistered [ 4841.548567] Key type ._llcrypt registered [ 4841.550964] Key type .llcrypt registered [ 4841.729271] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 4842.790064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4842.880340] alg: No test for adler32 (adler32-zlib) [ 4844.122941] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 4844.685387] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 4846.496487] Key type lgssc registered [ 4848.145956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4858.409800] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4858.468749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4860.251581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4864.868947] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4869.623467] LustreError: 147502:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4869.654444] LustreError: 147502:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4874.689495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4874.722764] LustreError: 147503:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4874.963479] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4879.591418] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4889.001983] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4895.673159] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4904.168874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4904.713371] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4905.726693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4913.587482] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 4920.104033] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4924.465255] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4931.378645] Lustre: *** cfs_fail_loc=159, val=0*** [ 4931.393260] Lustre: Skipped 1 previous similar message [ 4948.034122] Lustre: lustre-MDT0000: Client 794fe491-2905-448a-abae-d0978bd8da1d (at 192.168.204.18@tcp) reconnecting [ 4956.129976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4956.136594] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4956.143829] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4956.164168] Lustre: Skipped 1 previous similar message [ 4960.708771] Lustre: server umount lustre-OST0000 complete [ 4966.381815] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4966.389778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4966.393883] Lustre: Skipped 1 previous similar message [ 4966.405777] Lustre: Skipped 2 previous similar messages [ 4971.490043] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4971.500176] Lustre: Skipped 1 previous similar message [ 4972.573774] Lustre: server umount lustre-MDT0000 complete [ 4975.587685] LustreError: 147502:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4975.615360] LustreError: 147502:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4978.484118] LustreError: 148243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786951940 with bad export cookie 4801162241907620525 [ 4978.503572] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4978.513634] LustreError: 148243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4979.395933] Lustre: server umount lustre-MDT0001 complete [ 4997.697322] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 5001.428909] Key type lgssc unregistered [ 5001.844718] LNet: 151175:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5001.852240] LNetError: 151175:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5001.877963] LNet: Removed LNI 192.168.204.118@tcp [ 5002.937597] Key type .llcrypt unregistered [ 5002.941763] Key type ._llcrypt unregistered [ 5032.656495] Key type ._llcrypt registered [ 5032.658245] Key type .llcrypt registered [ 5032.735776] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 5033.974716] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5033.987874] alg: No test for adler32 (adler32-zlib) [ 5035.452902] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 5035.921507] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 5037.680190] Key type lgssc registered [ 5039.635336] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5048.335118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5048.364512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5050.088856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5054.968227] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5059.557052] LustreError: 152593:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5059.591842] LustreError: 152593:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5064.678424] LustreError: 152594:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5065.549472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5066.182427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5071.112257] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5079.496653] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5087.488753] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5095.759864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5096.196918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5102.961197] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5111.571845] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5113.668413] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5119.752111] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5140.924782] Lustre: lustre-MDT0001: evicting b9d5c4f3-e4ca-4a11-a771-d3567b3ae3ff (at 192.168.204.18@tcp) by administrative request [ 5140.935135] Lustre: Skipped 1 previous similar message [ 5171.172907] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5171.173097] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5171.199987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5171.212342] Lustre: Skipped 1 previous similar message [ 5176.198711] Lustre: server umount lustre-OST0000 complete [ 5181.409306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5181.411537] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5181.429652] Lustre: Skipped 2 previous similar messages [ 5186.530867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5186.534213] Lustre: Skipped 2 previous similar messages [ 5190.627155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5190.637504] Lustre: Skipped 1 previous similar message [ 5195.232146] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5195.365685] Lustre: server umount lustre-MDT0000 complete [ 5195.746629] LustreError: 152594:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5195.770831] LustreError: 152594:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5199.541034] LustreError: 152574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786952161 with bad export cookie 11747210559317198265 [ 5199.549667] LustreError: 152574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5199.550643] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5200.012837] Lustre: server umount lustre-MDT0001 complete [ 5212.971242] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 5216.108914] Key type lgssc unregistered [ 5216.462984] LNet: 156668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5216.477146] LNetError: 156668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5216.508296] LNet: Removed LNI 192.168.204.118@tcp [ 5217.407530] Key type .llcrypt unregistered [ 5217.411771] Key type ._llcrypt unregistered [ 5307.316872] Key type ._llcrypt registered [ 5307.320698] Key type .llcrypt registered [ 5307.469244] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 5308.656656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5308.761756] alg: No test for adler32 (adler32-zlib) [ 5310.050366] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 5310.285047] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 5312.052278] Key type lgssc registered [ 5313.431302] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5323.560738] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5323.574251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5325.170351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5330.742849] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5334.505967] LustreError: 159159:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5334.536483] LustreError: 159159:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5339.623664] LustreError: 159160:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5340.461492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5340.898802] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5346.763425] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5356.591281] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5364.120484] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5371.782379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5372.341565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5378.724520] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5385.679244] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5386.525920] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5394.881177] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5405.163777] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.172162] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5405.187302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5410.301605] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5410.305280] Lustre: Skipped 1 previous similar message [ 5411.281596] Lustre: server umount lustre-OST0000 complete [ 5415.394962] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5415.404696] Lustre: Skipped 1 previous similar message [ 5415.409089] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5415.415471] Lustre: Skipped 2 previous similar messages [ 5420.513868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5420.521108] Lustre: Skipped 1 previous similar message [ 5421.567197] Lustre: server umount lustre-MDT0000 complete [ 5425.480283] LustreError: 159141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786952387 with bad export cookie 4636953150550802668 [ 5425.485174] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5425.492705] LustreError: 159141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5425.768630] Lustre: server umount lustre-MDT0001 complete [ 5437.920801] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 5441.264947] Key type lgssc unregistered [ 5441.801242] LNet: 162730:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5441.814480] LNetError: 162730:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5441.837598] LNet: Removed LNI 192.168.204.118@tcp [ 5442.916403] Key type .llcrypt unregistered [ 5442.918480] Key type ._llcrypt unregistered [ 5474.255427] Key type ._llcrypt registered [ 5474.267711] Key type .llcrypt registered [ 5474.532336] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 5476.908508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5476.935836] alg: No test for adler32 (adler32-zlib) [ 5478.296478] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 5478.583282] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 5480.400856] Key type lgssc registered [ 5483.451909] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5501.234478] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 03:41:00 (1786952460) [ 5522.234797] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 5534.454199] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 5578.371794] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5578.415932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5579.775800] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5579.803455] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5579.901669] Lustre: lustre-MDT0000: new disk, initializing [ 5579.995418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5580.012923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5585.443713] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5597.598254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5597.827773] Lustre: 167016:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5597.949465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5597.953084] Lustre: Skipped 1 previous similar message [ 5598.091409] Lustre: lustre-MDT0001: new disk, initializing [ 5598.273893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5598.334533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5598.349293] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5604.034132] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5614.556421] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5620.936446] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5629.582763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5629.822097] Lustre: lustre-OST0000: new disk, initializing [ 5629.827539] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5629.838605] Lustre: 168992:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5629.952994] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5636.666916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5636.674529] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5636.808357] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5636.873094] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5645.985941] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5657.572530] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5657.577771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5657.590252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5657.592625] Lustre: Skipped 1 previous similar message [ 5657.604342] Lustre: Skipped 1 previous similar message [ 5662.699445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5662.711275] Lustre: Skipped 1 previous similar message [ 5662.930100] Lustre: server umount lustre-OST0000 complete [ 5667.825825] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5667.840731] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5667.861594] Lustre: Skipped 1 previous similar message [ 5667.881025] Lustre: Skipped 1 previous similar message [ 5672.936405] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5672.951065] Lustre: Skipped 1 previous similar message [ 5673.764688] Lustre: server umount lustre-MDT0000 complete [ 5678.235472] LustreError: 170150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786952639 with bad export cookie 7956334565671527797 [ 5678.238532] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5678.248794] LustreError: 170150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5678.631850] Lustre: server umount lustre-MDT0001 complete [ 5689.748760] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 5693.158734] Key type lgssc unregistered [ 5693.641620] LNet: 170903:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5693.652524] LNetError: 170903:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5693.678870] LNet: Removed LNI 192.168.204.118@tcp [ 5694.715916] Key type .llcrypt unregistered [ 5694.717837] Key type ._llcrypt unregistered [ 5724.947767] Key type ._llcrypt registered [ 5724.949901] Key type .llcrypt registered [ 5725.067879] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 5726.272445] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5726.291353] alg: No test for adler32 (adler32-zlib) [ 5727.608282] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 5728.024257] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 5729.784205] Key type lgssc registered [ 5731.279378] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5741.329111] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5741.343726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5742.772474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5747.353757] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5753.331309] LustreError: 172319:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5753.363841] LustreError: 172319:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5758.451202] LustreError: 172318:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5758.791537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5759.309679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5765.086446] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5774.559920] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5781.775315] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5789.192604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5789.622455] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5796.830330] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 5805.086425] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5813.210913] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5822.384116] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5822.425985] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5822.452476] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5822.481311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.499448] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5823.571438] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5823.607228] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5823.628080] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5823.642492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.661715] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5824.760928] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5824.788238] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5824.803808] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5824.813894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.822410] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5825.949909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.167788] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5827.184990] Lustre: Skipped 1 previous similar message [ 5827.213084] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5827.230678] Lustre: Skipped 1 previous similar message [ 5827.238823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5827.243138] Lustre: Skipped 1 previous similar message [ 5827.250037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.287342] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5827.292408] Lustre: Skipped 1 previous similar message [ 5828.440730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.745938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.998098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.014821] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5832.021934] Lustre: Skipped 3 previous similar messages [ 5832.046047] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5832.059298] Lustre: Skipped 3 previous similar messages [ 5832.064214] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5832.073738] Lustre: Skipped 3 previous similar messages [ 5832.082395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.095159] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5832.106929] Lustre: Skipped 3 previous similar messages [ 5833.180050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.338549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.446822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.439872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.604761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.655122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.685251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.825670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5840.838877] Lustre: Skipped 7 previous similar messages [ 5840.856574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5840.866387] Lustre: Skipped 7 previous similar messages [ 5840.869859] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5840.874179] Lustre: Skipped 7 previous similar messages [ 5840.878291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.889364] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5840.891671] Lustre: Skipped 7 previous similar messages [ 5841.936895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.897509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.826682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.776540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.915378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.963420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.983698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.894083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.003596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.101588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.061614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.010737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.848148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.829451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.861068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.779206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.764327] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5857.775181] Lustre: Skipped 16 previous similar messages [ 5857.808682] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5857.820190] Lustre: Skipped 16 previous similar messages [ 5857.825879] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5857.831050] Lustre: Skipped 16 previous similar messages [ 5857.837580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.853546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5857.862090] Lustre: Skipped 16 previous similar messages [ 5858.828949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.814033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.778694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.821755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.892334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.857330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.875748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.920659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.950252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.810297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.678508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.714912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.810930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.801815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.821588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.733136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.819657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.670346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.550233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.445188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.328759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.249927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.336065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.316505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.214563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.153692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.222880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.158495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.216883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.195085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.220172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.328782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.300652] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5890.317923] Lustre: Skipped 32 previous similar messages [ 5890.331918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5890.340886] Lustre: Skipped 32 previous similar messages [ 5890.344619] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5890.348587] Lustre: Skipped 32 previous similar messages [ 5890.351664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.362308] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5890.381551] Lustre: Skipped 32 previous similar messages [ 5891.263884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.273382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.304093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.252681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.211249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.274060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.158836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.134692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.129406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.327878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.132533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.065745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.924426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.898182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.811047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.738555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.644310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.585330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.345238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.356163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.422457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.344254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.402356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.537204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.468702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.678663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.763764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.782451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.787301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.639619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.630652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.663445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.660224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.762781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.857202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.765828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.949247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.977304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.917696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.016608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.907568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.839747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.755252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.729671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.729692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.689526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.743141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.737805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.826607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.994166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.006273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.926149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.878143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.855790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.688844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.580444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.487218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.453163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.264701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.178574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.023609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.850625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.586742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.639938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.433871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.267315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.201364] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5955.205186] Lustre: Skipped 66 previous similar messages [ 5955.218439] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5955.226208] Lustre: Skipped 66 previous similar messages [ 5955.229844] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5955.235282] Lustre: Skipped 66 previous similar messages [ 5955.238131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.248348] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5955.253026] Lustre: Skipped 66 previous similar messages [ 5955.978937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.821315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.633043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.470261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.272956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.971814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.832962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.728920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.452777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.341947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.391079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.238249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.166922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.257948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.159638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.970237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.922284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.102873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.008905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.080707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.974940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.026219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.063187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.921907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.934249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.943765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.014984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.094847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.094397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.104141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.064055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.022676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.951514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.923488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.779588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.620721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.575718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.506201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.375432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.240560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.154642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.110924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.988077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.796767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.588902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.653990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.519136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.587948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.423132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.137476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.147104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.007058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.948626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.899658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.840569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.568928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.404260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.211146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.101477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.852331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.620399] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6010.632614] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6010.635896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.417496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.237620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.089290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.110405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.927584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.938978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.076564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.070128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.072109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.981540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.896424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.810651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.779658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.955662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.308597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.550647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.472967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.804660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.836679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.448591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.894205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.224457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.382735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.342674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.474570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.899381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.920110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.025255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.143730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.158679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.427892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.636214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.634814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.004429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.177498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.119545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.208298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.290237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.637426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.545515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.682550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.839818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.142180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.355375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.343777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.424445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.258696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.246950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.223771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.035400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.963935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.060886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.418733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.382702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.457916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.698296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.727685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.618186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.608358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.637994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.858695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.910231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.059594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.300410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.519585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.876133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.199359] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6084.202819] Lustre: Skipped 127 previous similar messages [ 6084.254792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6084.277731] Lustre: Skipped 127 previous similar messages [ 6084.291290] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6084.304393] Lustre: Skipped 127 previous similar messages [ 6084.316815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.325208] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6084.327901] Lustre: Skipped 127 previous similar messages [ 6085.554368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.753567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.174241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.456348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.734096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.145261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.204440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.171556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.375757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.441568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.563170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.814155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.674310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.748533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.854157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.901610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.854654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.769770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.830486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.860349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.989366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.120309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.953841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.767233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.617315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.504436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.470856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.294876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.290309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.292503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.628702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.703630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.587308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.403479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.281635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.255992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.251702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.253540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.409790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.300169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.247208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.233044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.187623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.211875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.336846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.317164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.221786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.260786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.334743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.193858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.113924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.859188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.728911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.508709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.400044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.377466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.386100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.321506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.381149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.243739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.173735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.335250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.359932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.230988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.111599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.961497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.947163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.829891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.868996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.762647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.806616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.714820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.651169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.715712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.554390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.452198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.376407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.331506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.142092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.155579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.262854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.136444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.141750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.042134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.100480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.349837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.639989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.890203] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6172.903786] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6172.916145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.365994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.549551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.760100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.755248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.963988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.991172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.310970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.465364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.821522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.825021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.929949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.938968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.917523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.093971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.996042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.967641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.045785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.087470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.079478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.154406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.190756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.342570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.379380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.328083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.354251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.655187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.861919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.819830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.919908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.893485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.020176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.905288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.945270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.898894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.878423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.948532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.014390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.994060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.188501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.103524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.292599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.709485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.859322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.041051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.136223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.161506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.005734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.980555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.893181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.785386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.774608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.827209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.278165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.677884] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6231.700956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.701502] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6232.798407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.749228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.767922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.881426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.781735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.735748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.721639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.825586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.783763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.735580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.572732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.641024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.489808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.610065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.421912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.375455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.173205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.174144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.133740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.161699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.160995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.252438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.342130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.433943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.348326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.354825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.462285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.489819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.502231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.570681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.413271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.340167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.418639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.375604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.311460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.349580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.310752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.250342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.088096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.095067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.973083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.995914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.974569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.861135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.928439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.874844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.891529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.705125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.617735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.632320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.691733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.699516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.703268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.540686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.642478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.536885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.625956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.600715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.786560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.660473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.566045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.467829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.441563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.635405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.716565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.883708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.794619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.900435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.829399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.245727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.670956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.147115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.857275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.097301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.238486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.171189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.278133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.260314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.579540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.786334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.400080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.514300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.547543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.879561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.900978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.783914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.858438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.744703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.744293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.908174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.033575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.255885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.270521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.359276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.220293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.351684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.352816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.450180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.672698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.869659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.093475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.138296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.467210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.559626] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6340.563234] Lustre: Skipped 245 previous similar messages [ 6340.599548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6340.612838] Lustre: Skipped 245 previous similar messages [ 6340.619090] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6340.625696] Lustre: Skipped 245 previous similar messages [ 6340.632077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.644841] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6340.655319] Lustre: Skipped 245 previous similar messages [ 6341.771633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.914172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.150834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.269586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.234325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.302093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.304817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.309391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.301587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.337650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.544337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.650091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.752683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.957146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.876315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.808526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.824414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.742075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.900369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.758510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.796258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.774600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.977685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.925841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.877527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.704525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.831885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.818699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.706264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.553552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.362440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.348112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.400051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.625036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.633035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.574575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.385122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.568658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.534195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.711096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.817451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.817425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.997236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.096748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.088360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.288057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.392114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.722738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.731316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.991920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.314072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.296276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.492218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.487779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.724534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.990625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.320599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.281040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.368857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.311143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.276464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.146144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.112856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.038264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.054431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.133864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.308228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.393304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.438558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.295277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.258440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.312789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.373514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.350172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.292848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.262062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.141403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.027824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.925542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.967563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.803086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.820506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.867420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.878151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.714726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.895651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.815211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.784224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.810140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.877893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.704054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.827850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.169244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.233279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.098729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.038047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.096514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.298977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.264294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.507844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.009290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.211256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.439045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.987832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.468477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.946084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.416816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.957342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.477703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.740356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.727118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.991908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.406763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.426719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.743969] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6464.761314] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6464.793702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.958417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.117345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.197400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.267827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.517174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.912233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.035378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.270278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.283237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.474697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.455609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.549037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.790960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.272287] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6481.293627] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6481.319680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.511593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.563159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.822116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.981877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.984233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.047951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.212764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.246716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.470758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.634928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.659577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.858202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.109970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.255760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.483302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.681941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.564811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.546139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.439897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.556131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.501837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.324809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.185867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.211513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.119303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.152412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.130791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.362384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.648430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.517982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.516769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.536508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.723071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.820234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.862957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.152285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.363895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.809371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.202467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.656885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.217179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.790194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.082117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.817130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.369122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.917397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.312707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.591450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.450112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.625329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.279823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.123087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.215661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.438606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.612601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.802387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.827993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.799403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.765321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.985695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.027199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.024551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.248315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.450076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.509945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.746109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.834512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.123097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.263867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.410816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.506826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.771599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.064632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.306173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.188211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.197928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.465347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.581157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.562718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.649806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.872077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.980892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.327339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.523218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.425731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.438581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.382850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.379251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.407958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.444744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.338560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.350588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.181692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.489757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.458142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.622643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.109658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.218148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.162656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.419175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.797593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.428502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.657778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.784209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.217827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.853781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.733811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.188066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.313770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.655511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.875266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.274802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.105726] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6614.120345] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6614.137544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.458330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.914844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.303594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.600154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.804514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.075586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.256198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.298837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.595506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.844791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.091704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.450774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.694682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.707431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.840922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.989831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.253642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.531128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.914035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.052849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.152071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.253703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.315402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.459516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.410763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.388490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.525063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.748487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.618506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.731706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.830487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.762947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.726566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.553976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.340670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.307962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.276157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.400301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.340848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.564403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.642274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.892367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.982867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.394772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.940348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.239957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.542486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.734941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.131273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.756637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.162378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.555715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.627814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.110994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.521584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.083265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.559515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.791908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.793642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.841138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.045395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.917583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.734431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.744961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.720665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.671491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.798970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.783689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.931147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.748380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.048083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.903674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.728882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.595237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.010806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.972962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.846364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.803273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.681919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.594606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.545895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.519986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.466656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.378288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.271631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.503931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.360955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.358753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.274497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.327556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.395709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.483780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.498726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.655438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.937293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.058575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.217581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.374715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.578726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.453957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.558922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.541745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.501697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.516214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.578540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.488889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.481804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.351235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.347882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.318244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.326995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.180398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.148194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.258977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.325557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.277920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.324376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.254752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.140206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.936474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.924940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.778551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.702085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.661520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.629680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.633197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.583134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.688896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.643437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.679952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.692261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.646143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.780476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.828134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.069782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.115671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.184942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.166458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.125407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.078224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.097850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.401570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.617526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.663966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.024299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.058180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.008379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.873230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.078355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.990892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.080137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.021606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.064186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.272631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.187057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.222861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.267738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.688279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.999889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.529110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.647870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.022204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.342848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.021639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.523132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.738668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.877911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.175080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.667701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.044785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.257304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.420704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.609987] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6805.628606] LustreError: 172338:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6805.637733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.851668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.998623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.219531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.530495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.541978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.090345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.230948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.209531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.421142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.538513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.752457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.219943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.251755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.239348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.449305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.455555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.550658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.540857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.616278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.570884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.006953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.144947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.095621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.309412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.198688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.462551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.332789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.542769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.791349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.942992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.015753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.182481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.327531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.502468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.498787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.642582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.655068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.894560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.081904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.340169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.346694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.374260] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6853.379794] Lustre: Skipped 456 previous similar messages [ 6853.398973] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6853.438591] Lustre: Skipped 456 previous similar messages [ 6853.450125] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6853.460665] Lustre: Skipped 456 previous similar messages [ 6853.468668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.499163] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6853.505410] Lustre: Skipped 456 previous similar messages [ 6854.505784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.787844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.337503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.910978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.098958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.541567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.952341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.280629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.688937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.197638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.986406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.517355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.339347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.285189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.329531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.687355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.844588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.344678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.489283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.606249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.640640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.862591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.226363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.610698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.691562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.100654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.403577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.723333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.994903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.113394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.222940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.436093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.541143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.907971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.286900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.868593] LustreError: 174143:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7433.890119] LustreError: 172338:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8195.044128] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8195.056242] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8195.062477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8195.094710] Lustre: Skipped 36 previous similar messages [ 8199.641786] Lustre: server umount lustre-OST0000 complete [ 8205.288132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8205.297239] Lustre: Skipped 2 previous similar messages [ 8210.402913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8210.410923] Lustre: Skipped 1 previous similar message [ 8214.497829] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8214.513215] Lustre: Skipped 1 previous similar message [ 8218.083575] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8218.602990] Lustre: server umount lustre-MDT0000 complete [ 8219.634906] LustreError: 173047:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8219.675979] LustreError: 173047:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8222.807057] LustreError: 174165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786955184 with bad export cookie 4135905781003197963 [ 8222.817423] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8222.827303] LustreError: 174165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8223.236148] Lustre: server umount lustre-MDT0001 complete [ 8236.747421] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 8239.916914] Key type lgssc unregistered [ 8240.358848] LNet: 227668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8240.371497] LNetError: 227668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8241.448076] LNet: Removed LNI 192.168.204.118@tcp [ 8242.658591] Key type .llcrypt unregistered [ 8242.662220] Key type ._llcrypt unregistered [ 8273.388227] Key type ._llcrypt registered [ 8273.391526] Key type .llcrypt registered [ 8273.531741] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 8275.112413] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8275.186076] alg: No test for adler32 (adler32-zlib) [ 8276.500375] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 8276.877495] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 8278.568207] Key type lgssc registered [ 8280.805248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8290.678906] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8290.711911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.287223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8298.050693] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8301.540074] LustreError: 229088:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8301.578558] LustreError: 229088:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8306.658570] LustreError: 229087:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8309.079213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8309.444375] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8314.978209] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8324.108875] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8331.479286] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8339.438352] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8339.892473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8347.074645] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8355.707414] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8362.431462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8365.992350] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8368.525754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8368.546324] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8368.554514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8368.569661] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8368.571307] LustreError: 229107:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8372.710178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8372.740538] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8372.751904] Lustre: Skipped 1 previous similar message [ 8372.770551] Lustre: Skipped 1 previous similar message [ 8377.380764] Lustre: server umount lustre-OST0000 complete [ 8388.576194] LustreError: 229107:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8414.181368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8414.828245] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8422.087622] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8431.806550] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8438.330760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8438.352693] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8438.359654] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8438.389037] Lustre: Skipped 1 previous similar message [ 8439.225224] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 5 sec [ 8446.654408] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cbf85f02000.ost_server_uuid 50 [ 8448.663528] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cbf85f02000.ost_server_uuid in FULL state after 0 sec [ 8458.725699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8458.730095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8458.753258] Lustre: Skipped 1 previous similar message [ 8458.765439] Lustre: Skipped 1 previous similar message [ 8460.568297] Lustre: server umount lustre-OST0000 complete [ 8468.449333] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8468.459675] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8468.480392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8468.486967] Lustre: Skipped 1 previous similar message [ 8472.602869] Lustre: server umount lustre-MDT0000 complete [ 8474.092299] LustreError: 229088:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8477.940956] LustreError: 229069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786955439 with bad export cookie 2201262902104127946 [ 8477.970836] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8478.533863] Lustre: server umount lustre-MDT0001 complete [ 8493.963635] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 8497.441989] Key type lgssc unregistered [ 8497.760693] LNet: 234305:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8497.772789] LNetError: 234305:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8498.798321] LNet: Removed LNI 192.168.204.118@tcp [ 8499.895126] Key type .llcrypt unregistered [ 8499.901982] Key type ._llcrypt unregistered [ 8537.994048] Key type ._llcrypt registered [ 8537.999759] Key type .llcrypt registered [ 8538.209479] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 8540.143096] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8540.182459] alg: No test for adler32 (adler32-zlib) [ 8541.287155] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 8541.507895] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 8543.137088] Key type lgssc registered [ 8544.656692] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8555.679369] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8555.709507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8557.269476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8562.725274] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8566.755095] LustreError: 236044:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8566.778276] LustreError: 236044:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8571.877439] LustreError: 236043:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8572.214766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8572.782648] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8578.409563] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8582.140840] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8590.675884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8591.037879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8599.146636] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8602.306097] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8609.267488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8609.498947] Lustre: lustre-OST0001: new disk, initializing [ 8609.502976] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8609.509581] Lustre: 238358:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8609.625890] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8615.558529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8615.577242] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8615.678106] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8616.741626] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8628.645258] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8634.260052] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8649.701107] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8649.717378] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8649.738179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8651.755680] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8651.758737] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8651.768571] Lustre: Skipped 1 previous similar message [ 8651.790648] Lustre: Skipped 3 previous similar messages [ 8654.074990] Lustre: server umount lustre-MDT0000 complete [ 8656.876397] LustreError: 236043:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8656.907557] LustreError: 236043:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8658.303586] LustreError: 238356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786955619 with bad export cookie 4419654826746941902 [ 8658.311709] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8658.320861] LustreError: 238356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8658.828877] Lustre: server umount lustre-MDT0001 complete [ 8674.032016] Lustre: server umount lustre-OST0000 complete [ 8688.327587] Lustre: server umount lustre-OST0001 complete [ 8713.344216] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 8725.941118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8726.685289] LustreError: 241469:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8726.848209] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8732.129537] LustreError: 241470:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8732.854142] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8742.381739] LustreError: 241469:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8742.420214] LustreError: 241469:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8742.863421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8743.561732] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8748.403848] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8752.097902] Lustre: 242610:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8759.879453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8760.287951] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8767.437742] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8767.472695] LustreError: 242965:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8771.600725] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8778.162783] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8785.640694] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8794.604499] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8798.542518] Lustre: 244481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8814.562772] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8814.567028] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8814.615175] Lustre: Skipped 3 previous similar messages [ 8814.626371] Lustre: Skipped 1 previous similar message [ 8816.875747] Lustre: server umount mylustre-MDT0000 complete [ 8819.698661] LustreError: 242612:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8819.729471] LustreError: 242612:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 8821.369418] LustreError: 241447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786955782 with bad export cookie 4419654826746944709 [ 8821.374023] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8821.383176] LustreError: 241447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8821.730228] Lustre: server umount mylustre-MDT0001 complete [ 8836.402499] Lustre: server umount mylustre-OST0000 complete [ 8851.131665] Lustre: server umount mylustre-OST0001 complete [ 8878.113561] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 8889.610708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8890.140978] LustreError: 246793:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8890.299952] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8890.310877] Lustre: Skipped 1 previous similar message [ 8895.805915] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8905.735432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8910.646722] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8913.893401] Lustre: 247938:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8921.314208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8930.426908] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8935.104463] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 8942.592988] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8948.239864] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8950.499670] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 8960.146658] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8964.685749] Lustre: 249809:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8977.892159] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8977.902740] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8977.921506] Lustre: Skipped 1 previous similar message [ 8977.939335] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8977.960128] Lustre: Skipped 2 previous similar messages [ 8981.598947] Lustre: server umount tfs-MDT0000 complete [ 8985.602610] LustreError: 246770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786955947 with bad export cookie 4419654826746946375 [ 8985.609933] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8985.616844] LustreError: 246770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8985.984415] Lustre: server umount tfs-MDT0001 complete [ 9001.932669] Lustre: server umount tfs-OST0000 complete [ 9004.704253] Lustre: 235305:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786955950/real 1786955950] req@ffff91d3dfb5ad80 x1873758653732608/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786955966 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9004.737764] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9004.760745] Lustre: Skipped 3 previous similar messages [ 9006.264636] Lustre: server umount tfs-OST0001 complete [ 9035.444884] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9047.729392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9048.289786] LustreError: 252075:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9048.317699] LustreError: 252075:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 9048.379295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9048.383504] Lustre: Skipped 3 previous similar messages [ 9053.920579] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9063.425247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9069.191502] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9072.709314] Lustre: 253218:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9080.779235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9089.007592] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9092.633249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 9098.714477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9104.431393] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 9107.049662] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9116.254814] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9120.567311] Lustre: 255087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9129.650880] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 04:41:28 (1786956088) [ 9150.438521] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9150.442989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9150.444894] Lustre: Skipped 3 previous similar messages [ 9150.448966] Lustre: Skipped 7 previous similar messages [ 9152.778480] Lustre: server umount lustre-MDT0000 complete [ 9157.452419] LustreError: 252051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786956119 with bad export cookie 4419654826746948034 [ 9157.461174] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9157.473450] LustreError: 252051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9160.693906] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9160.709824] Lustre: Skipped 1 previous similar message [ 9164.342800] Lustre: server umount lustre-MDT0001 complete [ 9176.295580] Lustre: server umount lustre-OST0000 complete [ 9187.958497] Lustre: server umount lustre-OST0001 complete [ 9195.430103] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 9212.367810] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9256.581408] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 9256.581408] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9256.581408] [ 9256.623387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9256.965943] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9257.011170] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9257.113805] Lustre: lustre-MDT0000: new disk, initializing [ 9257.221640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9257.231181] Lustre: Skipped 3 previous similar messages [ 9257.266298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9262.448822] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9274.418231] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 9274.418231] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 9274.418231] [ 9274.469805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 9274.605350] Lustre: 258298:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9274.660135] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9274.666746] Lustre: Skipped 1 previous similar message [ 9274.762542] Lustre: lustre-MDT0001: new disk, initializing [ 9274.951077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9274.968765] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9280.392597] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9293.095709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9293.406684] Lustre: lustre-OST0000: new disk, initializing [ 9293.418568] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9293.431423] Lustre: 259886:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9295.463635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9295.484540] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9295.647085] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9301.260984] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9312.081023] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9312.234467] Lustre: lustre-OST0001: new disk, initializing [ 9312.237642] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9312.242743] Lustre: 260754:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9314.250577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9314.256711] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9314.359093] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9318.386587] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9330.277238] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 04:44:49 (1786956289) [ 9346.351633] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 04:45:05 (1786956305) [ 9362.401482] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9362.412340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9362.414242] LustreError: Skipped 1 previous similar message [ 9362.417316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9362.430070] Lustre: Skipped 3 previous similar messages [ 9362.442527] Lustre: Skipped 3 previous similar messages [ 9367.453418] Lustre: server umount lustre-OST0000 complete [ 9368.581563] LustreError: 259867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9368.615326] LustreError: 259867:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 9372.129845] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9378.501276] Lustre: server umount lustre-MDT0000 complete [ 9382.935178] LustreError: 258291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786956344 with bad export cookie 4419654826746949595 [ 9382.938613] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9382.953491] LustreError: 258291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9389.404488] Lustre: server umount lustre-MDT0001 complete [ 9399.922418] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 9472.185460] Lustre: server umount lustre-OST0001 complete [ 9479.542428] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [ 9496.338859] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9532.720741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9532.961322] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9532.987736] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9533.098830] Lustre: lustre-MDT0000: new disk, initializing [ 9533.174617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9533.179358] Lustre: Skipped 3 previous similar messages [ 9533.189952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9538.002106] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9549.311832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9549.438954] Lustre: 265552:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9549.488280] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9549.498352] Lustre: Skipped 1 previous similar message [ 9549.560832] Lustre: lustre-MDT0001: new disk, initializing [ 9549.697827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9549.714376] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9554.435476] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9564.501586] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9570.603888] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9577.701490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9578.024263] Lustre: lustre-OST0000: new disk, initializing [ 9578.038372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9578.050086] Lustre: 267525:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9579.567436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9579.575528] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9579.663423] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9584.930430] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9595.605216] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9605.092705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9605.100947] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9605.110064] Lustre: Skipped 4 previous similar messages [ 9605.111753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9605.136901] Lustre: Skipped 9 previous similar messages [ 9608.889291] Lustre: server umount lustre-OST0000 complete [ 9618.788168] Lustre: server umount lustre-MDT0000 complete [ 9622.298128] LustreError: 265544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786956583 with bad export cookie 4419654826746951065 [ 9622.304179] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9622.309813] LustreError: 265544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9622.725941] Lustre: server umount lustre-MDT0001 complete [ 9634.170341] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 9637.109733] Key type lgssc unregistered [ 9637.514331] LNet: 269288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9637.520694] LNetError: 269288:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9637.544683] LNet: Removed LNI 192.168.204.118@tcp [ 9638.649161] Key type .llcrypt unregistered [ 9638.653736] Key type ._llcrypt unregistered [ 9670.335563] Key type ._llcrypt registered [ 9670.337800] Key type .llcrypt registered [ 9670.430439] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9671.579883] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9671.751677] alg: No test for adler32 (adler32-zlib) [ 9672.872730] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 9673.143986] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 9674.817805] Key type lgssc registered [ 9676.008782] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9683.234843] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9683.272849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9684.769270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9689.262408] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9695.201975] LustreError: 270753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9695.223158] LustreError: 270753:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9698.963732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9699.491708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9704.549420] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9713.198852] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9719.879653] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9727.230785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9727.634206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9729.743538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9734.454739] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9742.018783] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9743.012084] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9745.379690] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9745.391533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9745.412386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9749.474282] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9749.476498] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9749.479673] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9749.499136] Lustre: Skipped 1 previous similar message [ 9751.204102] Lustre: server umount lustre-OST0000 complete [ 9755.633305] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9755.642884] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9755.658187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9759.714916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9759.719412] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9761.595911] LustreError: 273586:0:(ldlm_resource.c:1208:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (00000000889e643d) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 9761.723222] Lustre: server umount lustre-MDT0000 complete [ 9764.836951] LustreError: 270753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9764.861258] LustreError: 270753:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9765.755806] LustreError: 270733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786956727 with bad export cookie 6315102438066497557 [ 9765.770957] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9765.771444] LustreError: 270733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9766.123320] Lustre: server umount lustre-MDT0001 complete [ 9778.175784] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 9782.160856] Key type lgssc unregistered [ 9782.562350] LNet: 274342:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9782.571931] LNetError: 274342:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9783.655485] LNet: Removed LNI 192.168.204.118@tcp [ 9784.866601] Key type .llcrypt unregistered [ 9784.868843] Key type ._llcrypt unregistered [ 9803.583433] Key type ._llcrypt registered [ 9803.586376] Key type .llcrypt registered [ 9803.729358] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9805.340232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9805.369603] alg: No test for adler32 (adler32-zlib) [ 9806.430160] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 9806.666292] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 9808.384592] Key type lgssc registered [ 9809.406202] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9818.872254] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9818.897017] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9834.464505] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9839.584321] LustreError: 275538:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.118@tcp: failed processing log, type 4: rc = -110 [ 9866.272373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9872.830534] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [ 9880.634374] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9882.934376] Lustre: server umount lustre-OST0000 complete [ 9901.908644] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [ 9904.897346] Key type lgssc unregistered [ 9905.176959] LNet: 277176:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9905.188135] LNetError: 277176:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9905.210636] LNet: Removed LNI 192.168.204.118@tcp [ 9906.482759] Key type .llcrypt unregistered [ 9906.490642] Key type ._llcrypt unregistered [ 9948.050880] Key type ._llcrypt registered [ 9948.053612] Key type .llcrypt registered [ 9948.177446] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [ 9949.376637] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9949.534732] alg: No test for adler32 (adler32-zlib) [ 9950.722126] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [ 9951.008892] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [ 9952.704278] Key type lgssc registered [ 9954.153316] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9968.040634] loop0: detected capacity change from 0 to 204800000 [ 9968.234959] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9968.244679] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9968.570031] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9969.632159] loop0: detected capacity change from 0 to 204800000 [ 9969.816024] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9969.826863] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9970.165950] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9971.214845] loop0: detected capacity change from 0 to 204800000 [ 9971.384651] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9971.399418] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9971.550893] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9972.707843] loop0: detected capacity change from 0 to 204800000 [ 9972.858701] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9974.136510] loop0: detected capacity change from 0 to 204800000 [ 9974.166733] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9978.023939] loop0: detected capacity change from 0 to 204800000 [ 9978.045232] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9981.770446] loop0: detected capacity change from 0 to 204800000 [ 9981.804848] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9985.725431] loop0: detected capacity change from 0 to 204800000 [ 9985.758525] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9990.465860] loop0: detected capacity change from 0 to 204800000 [ 9990.549411] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9990.633499] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9990.707996] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9990.718866] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9990.734935] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9990.745957] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9990.755057] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9990.762765] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9990.769781] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9990.778359] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9990.790845] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9990.801560] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9990.807941] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9990.822519] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9990.834825] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9990.847553] Lustre: 280588:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9996.360622] Lustre: server umount MGS complete [ 9997.516695] loop0: detected capacity change from 0 to 204800000 [ 9997.618145] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9998.124760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9998.178514] Lustre: lustre-MDT0000: Aborting client recovery [ 9998.987138] loop1: detected capacity change from 0 to 204800000 [ 9999.118649] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9999.271028] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9999.285869] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9999.304198] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9999.314842] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9999.329112] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9999.342828] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9999.352112] Lustre: 281020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9999.370845] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9999.540230] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9999.604752] Lustre: lustre-MDT0001: Aborting client recovery [10000.735914] loop2: detected capacity change from 0 to 204800000 [10000.921283] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10001.059764] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10001.076898] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [10001.087220] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10001.098644] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10001.115655] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [10001.131312] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10001.143178] Lustre: 281098:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [10001.381849] Lustre: Found index 0 for lustre-OST0000, updating log [10001.448622] Lustre: 281117:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10001.542839] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10001.605494] Lustre: lustre-OST0000: Aborting recovery [10002.574893] loop3: detected capacity change from 0 to 204800000 [10002.709731] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10002.826731] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [10002.845866] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [10002.877344] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [10002.894309] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [10002.910803] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [10002.922941] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [10002.933850] Lustre: 281183:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [10002.960585] Lustre: Found index 1 for lustre-OST0001, updating log [10003.012777] Lustre: 281188:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10003.093927] Lustre: lustre-OST0001: Aborting recovery [10008.239077] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10008.272299] LustreError: 280964:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [10008.287404] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [10008.304497] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [10008.349411] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [10010.281512] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [10010.283368] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [10012.647080] LustreError: 280931:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 514us: rc = -2 [10013.163413] LustreError: 280929:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 598us: rc = -2 [10013.184908] LustreError: 280929:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [10030.561728] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10030.569219] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10030.582270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10031.073427] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10031.083314] Lustre: Skipped 2 previous similar messages [10031.087630] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10031.102024] Lustre: Skipped 3 previous similar messages [10033.072335] Lustre: server umount lustre-MDT0000 complete [10033.952428] LustreError: 280915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786956995 with bad export cookie 13553999322046889617 [10033.969484] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10036.192812] LustreError: 280929:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10036.193081] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10036.193500] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10036.206435] LustreError: 280929:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10036.241595] Lustre: Skipped 1 previous similar message [10038.244469] LustreError: 280930:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10038.246082] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10038.286381] Lustre: Skipped 1 previous similar message [10040.385270] Lustre: server umount lustre-MDT0001 complete [10047.480820] Lustre: server umount lustre-OST0000 complete [10054.464360] Lustre: server umount lustre-OST0001 complete [10062.433839] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 04:57:01 (1786957021) [10090.195691] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [10099.979594] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [10136.024571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10136.241683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10136.267834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10136.360408] Lustre: lustre-MDT0000: new disk, initializing [10136.443206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10136.449575] Lustre: Skipped 1 previous similar message [10136.462183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10140.865812] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10152.342664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10152.520396] Lustre: 285254:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10152.579730] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10152.583887] Lustre: Skipped 1 previous similar message [10152.662750] Lustre: lustre-MDT0001: new disk, initializing [10152.801080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10152.832281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10152.840585] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10157.494166] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10166.948123] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10172.431779] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10179.807594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10180.174393] Lustre: lustre-OST0000: new disk, initializing [10180.182709] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10180.197284] Lustre: 287222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10180.316121] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10186.820455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10186.834606] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10186.965312] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10187.344547] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10199.065481] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10204.154679] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10216.978847] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10220.950960] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10225.461579] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10274.278930] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10274.280564] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10274.280920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10274.290800] Lustre: Skipped 1 previous similar message [10279.704592] Lustre: server umount lustre-OST0000 complete [10284.521548] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10284.536248] Lustre: Skipped 1 previous similar message [10284.537813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10284.548804] Lustre: Skipped 4 previous similar messages [10289.580671] Lustre: server umount lustre-MDT0000 complete [10289.634795] LustreError: 285261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10289.651070] LustreError: 285261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10293.811328] LustreError: 287221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786957255 with bad export cookie 13553999322046892333 [10293.825333] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10293.827836] LustreError: 287221:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10294.143726] Lustre: server umount lustre-MDT0001 complete [10302.770217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10307.214328] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10312.147218] Lustre: server umount MGS complete [10323.346541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10323.921332] LustreError: 291049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10323.954603] LustreError: 291062:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10324.032670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10328.675502] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10329.058086] LustreError: 291050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10329.094618] LustreError: 291050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10336.687951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10341.324746] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10350.046275] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10356.273817] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10364.318588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10364.650395] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10364.659414] Lustre: Skipped 1 previous similar message [10371.642121] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10379.819173] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10413.027787] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10413.037990] LustreError: Skipped 1 previous similar message [10413.038060] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10413.053162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10413.057642] Lustre: Skipped 1 previous similar message [10416.931928] Lustre: server umount lustre-OST0000 complete [10427.536339] Lustre: server umount lustre-MDT0000 complete [10428.386597] LustreError: 291049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10428.431354] LustreError: 291049:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10431.798615] LustreError: 293776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786957393 with bad export cookie 13553999322046894160 [10431.805494] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10431.816802] LustreError: 293776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10432.300852] Lustre: server umount lustre-MDT0001 complete [10446.652402] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [10449.592862] Key type lgssc unregistered [10449.885090] LNet: 294999:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10449.898678] LNetError: 294999:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10449.930824] LNet: Removed LNI 192.168.204.118@tcp [10450.766773] Key type .llcrypt unregistered [10450.774125] Key type ._llcrypt unregistered [10491.375171] Key type ._llcrypt registered [10491.377885] Key type .llcrypt registered [10491.522976] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [10507.354158] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [10508.386544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10508.545514] alg: No test for adler32 (adler32-zlib) [10509.730313] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [10509.957333] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [10511.641491] Key type lgssc registered [10513.101370] Lustre: Echo OBD driver; http://www.lustre.org/ [10558.707287] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10558.761083] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10560.289201] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10560.331469] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10560.498150] Lustre: lustre-MDT0000: new disk, initializing [10560.615482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10560.644873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10566.836021] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10578.788644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10578.904302] Lustre: 298961:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10578.960540] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10578.966819] Lustre: Skipped 1 previous similar message [10579.063968] Lustre: lustre-MDT0001: new disk, initializing [10579.191209] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10579.219491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10579.237173] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10583.779385] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10594.194865] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10601.392097] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10609.282113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10609.691814] Lustre: lustre-OST0000: new disk, initializing [10609.697534] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10609.711724] Lustre: 300929:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10609.827190] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10616.570707] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10616.984490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10616.999377] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10617.063869] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10626.441130] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10633.047472] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10645.357568] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10652.776984] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10664.412147] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10719.720643] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10719.733458] Lustre: Skipped 1 previous similar message [10719.735240] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10719.735641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10719.749631] LustreError: Skipped 1 previous similar message [10720.893870] Lustre: server umount lustre-OST0000 complete [10724.836448] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10724.838939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10724.846965] Lustre: Skipped 1 previous similar message [10724.866928] Lustre: Skipped 2 previous similar messages [10729.957220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10729.962339] Lustre: Skipped 1 previous similar message [10730.448922] Lustre: server umount lustre-MDT0000 complete [10734.330909] LustreError: 301787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786957695 with bad export cookie 1894447983230053259 [10734.335432] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10734.344805] LustreError: 301787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10734.689841] Lustre: server umount lustre-MDT0001 complete [10743.197790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10748.332526] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10753.464610] Lustre: server umount MGS complete [10765.734532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10766.247231] LustreError: 304932:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10766.324028] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10771.172747] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10771.431497] LustreError: 304920:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10771.443875] LustreError: 304920:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10775.531348] LustreError: 304919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10780.427701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10780.642361] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10780.778099] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10785.603581] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10793.050485] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10798.197921] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10804.523962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10804.783188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10810.836659] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [10818.740282] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10851.810814] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10851.825915] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10851.852710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10851.867236] Lustre: Skipped 1 previous similar message [10854.434769] Lustre: server umount lustre-OST0000 complete [10862.057867] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10862.080806] Lustre: Skipped 1 previous similar message [10862.089438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10862.099765] Lustre: Skipped 1 previous similar message [10864.972649] Lustre: server umount lustre-MDT0000 complete [10867.172895] LustreError: 307482:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10867.197448] LustreError: 307482:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10868.880246] LustreError: 307648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786957830 with bad export cookie 1894447983230055198 [10868.885245] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10868.893033] LustreError: 307648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10869.319489] Lustre: server umount lustre-MDT0001 complete [10882.525340] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [10885.296694] Key type lgssc unregistered [10885.782189] LNet: 308869:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10885.796473] LNetError: 308869:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10885.825827] LNet: Removed LNI 192.168.204.118@tcp [10887.137403] Key type .llcrypt unregistered [10887.140540] Key type ._llcrypt unregistered [10915.527287] Key type ._llcrypt registered [10915.530087] Key type .llcrypt registered [10915.628211] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [10935.211589] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [10936.354606] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10936.548238] alg: No test for adler32 (adler32-zlib) [10937.632921] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [10937.932655] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [10939.640673] Key type lgssc registered [10940.957645] Lustre: Echo OBD driver; http://www.lustre.org/ [10968.483778] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [10984.446420] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10984.483415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10985.780979] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10985.804292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10985.920381] Lustre: lustre-MDT0000: new disk, initializing [10986.032715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10986.065990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10990.968243] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [11002.444952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11002.679417] Lustre: lustre-OST0000: new disk, initializing [11002.683403] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11002.687348] Lustre: Skipped 1 previous similar message [11002.692671] Lustre: 312921:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11002.753471] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11005.208513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11005.212860] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11005.258853] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11008.903665] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [11983.334472] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11983.347762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11988.543296] Lustre: server umount lustre-MDT0000 complete [11992.270487] LustreError: 312111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786958953 with bad export cookie 16381392505745173308 [11992.277938] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11992.281935] LustreError: 312111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11992.467258] Lustre: server umount lustre-OST0000 complete [12019.613276] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [12029.758602] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [12064.636077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12064.893193] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12064.928374] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12065.019829] Lustre: lustre-MDT0000: new disk, initializing [12065.119561] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12065.134571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12069.913354] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12082.922831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12083.078273] Lustre: 317275:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12083.167268] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12083.172046] Lustre: Skipped 1 previous similar message [12083.274940] Lustre: lustre-MDT0001: new disk, initializing [12083.392731] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12083.437029] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12083.445327] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12088.112186] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12098.681519] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12105.371178] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12112.469817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12112.806682] Lustre: lustre-OST0000: new disk, initializing [12112.811470] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12112.824952] Lustre: 319246:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12112.909772] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12119.364852] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12121.164258] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12121.180242] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12121.315985] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12130.756436] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12137.486163] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12137.832720] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12143.099589] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12143.375586] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12147.175770] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12147.181137] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12147.212033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12147.217732] Lustre: Skipped 1 previous similar message [12152.207570] Lustre: server umount lustre-OST0000 complete [12157.410457] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12157.427368] Lustre: Skipped 1 previous similar message [12157.431892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12157.438673] Lustre: Skipped 1 previous similar message [12162.529400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12162.543273] Lustre: Skipped 1 previous similar message [12163.408498] Lustre: server umount lustre-MDT0000 complete [12166.627337] LustreError: 317283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12166.655982] LustreError: 317283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12167.723141] LustreError: 318073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786959129 with bad export cookie 16381392505748114127 [12167.726250] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12167.733802] LustreError: 318073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12168.289604] Lustre: server umount lustre-MDT0001 complete [12172.285608] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12174.526414] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 05:32:13 (1786959133) [12181.555127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12182.178595] LustreError: 321560:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12182.261984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12187.631531] LustreError: 321561:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12187.735833] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12191.716212] LustreError: 321560:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12196.838069] LustreError: 321561:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12197.817709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12198.265705] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12203.332361] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12211.946893] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12218.978819] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12226.848342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12226.978469] LustreError: 323384:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12227.089498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12233.500208] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12242.139755] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12250.717164] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12250.950790] Lustre: lustre-OST0001: new disk, initializing [12250.954566] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12250.961725] Lustre: 324398:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12251.062450] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12257.854128] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12259.905831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12259.923459] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12259.983227] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12261.026616] LustreError: 324863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12261.044581] LustreError: 324863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12262.054090] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12264.103556] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12264.122278] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12267.910302] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12268.136539] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12268.268848] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12268.291133] LustreError: 324859:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [12273.945737] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cbfaba9b800.ost_server_uuid 50 [12275.531213] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cbfaba9b800.ost_server_uuid in FULL state after 0 sec [12276.459668] LustreError: 324863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12276.477021] LustreError: 324863:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [12282.276807] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9cbfaba9b800.ost_server_uuid 50 [12284.168783] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9cbfaba9b800.ost_server_uuid in FULL state after 0 sec [12285.756241] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12301.287197] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12301.293606] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12301.301829] LustreError: Skipped 1 previous similar message [12301.304759] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12301.326250] Lustre: Skipped 2 previous similar messages [12301.342952] Lustre: Skipped 1 previous similar message [12305.814871] Lustre: server umount lustre-OST0001 complete [12306.410284] LustreError: 323383:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12306.424465] LustreError: 323383:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12314.082559] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12314.094359] LustreError: Skipped 1 previous similar message [12314.102287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12314.129789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12314.137114] Lustre: Skipped 1 previous similar message [12318.619926] Lustre: server umount lustre-OST0000 complete [12326.381853] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12326.388943] LustreError: Skipped 1 previous similar message [12326.394292] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12326.407486] Lustre: Skipped 1 previous similar message [12329.691742] Lustre: server umount lustre-MDT0000 complete [12333.621206] LustreError: 321542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786959295 with bad export cookie 16381392505748115009 [12333.627539] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12333.634640] LustreError: 321542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12334.056527] Lustre: server umount lustre-MDT0001 complete [12348.923412] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [12352.177147] Key type lgssc unregistered [12352.553525] LNet: 327091:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12352.567891] LNetError: 327091:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12352.584078] LNet: Removed LNI 192.168.204.118@tcp [12353.520258] Key type .llcrypt unregistered [12353.523494] Key type ._llcrypt unregistered [12384.007406] Key type ._llcrypt registered [12384.013084] Key type .llcrypt registered [12384.093873] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [12384.930359] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12384.998463] alg: No test for adler32 (adler32-zlib) [12386.276473] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [12386.516098] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [12388.216389] Key type lgssc registered [12389.348485] Lustre: Echo OBD driver; http://www.lustre.org/ [12397.856658] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12397.885618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12399.369703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12404.054512] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12408.808483] LustreError: 328511:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12408.834569] LustreError: 328511:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12412.672053] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12413.924422] LustreError: 328512:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12419.050939] LustreError: 328511:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12421.053329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12421.547488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12426.533990] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12435.500326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12435.813053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12442.261546] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12444.005227] LustreError: 330192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12444.019017] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12449.897528] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12451.297770] LustreError: 330193:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12451.323474] LustreError: 330193:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12456.612706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12456.888165] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12462.073254] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12463.380195] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12470.973768] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12478.607196] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12478.819965] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12590.051539] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12590.066143] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12590.084036] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12590.093632] Lustre: Skipped 1 previous similar message [12595.172179] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12595.179789] Lustre: Skipped 1 previous similar message [12600.295571] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12602.338782] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12602.516643] Lustre: server umount lustre-OST0001 complete [12605.419379] LustreError: 330666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12605.448862] LustreError: 330666:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12610.552303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12610.574489] Lustre: Skipped 1 previous similar message [12610.584189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12610.589099] Lustre: Skipped 1 previous similar message [12615.139257] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12615.157678] LustreError: Skipped 1 previous similar message [12615.164669] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12615.188043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12616.406217] Lustre: server umount lustre-OST0000 complete [12621.285319] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12621.297238] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12625.892344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12625.895732] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12625.898242] Lustre: Skipped 2 previous similar messages [12627.448990] Lustre: server umount lustre-MDT0000 complete [12631.013120] LustreError: 329435:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12631.044959] LustreError: 329435:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12631.668907] LustreError: 328492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786959593 with bad export cookie 5812661480710373951 [12631.682071] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12631.689667] LustreError: 328492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12632.125923] Lustre: server umount lustre-MDT0001 complete [12644.314170] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [12647.367707] Key type lgssc unregistered [12647.690647] LNet: 333468:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12647.696546] LNetError: 333468:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12647.707421] LNet: Removed LNI 192.168.204.118@tcp [12648.544611] Key type .llcrypt unregistered [12648.549337] Key type ._llcrypt unregistered [12693.811317] Key type ._llcrypt registered [12693.813873] Key type .llcrypt registered [12693.982066] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [12695.063871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12695.159105] alg: No test for adler32 (adler32-zlib) [12696.348790] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [12696.534622] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [12698.200275] Key type lgssc registered [12699.617321] Lustre: Echo OBD driver; http://www.lustre.org/ [12709.775977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12709.819102] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12711.496322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12716.123857] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12721.636446] LustreError: 335754:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12721.650536] LustreError: 335754:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12725.411623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12725.948068] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12730.678683] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12733.652889] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12740.266658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12740.642926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12747.059565] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12750.886448] LustreError: 337244:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12755.698299] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12756.050061] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12757.121452] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12757.141193] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12762.809822] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12773.403546] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12779.820140] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12793.317909] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12793.326029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12793.343665] Lustre: Skipped 3 previous similar messages [12793.359944] Lustre: Skipped 3 previous similar messages [12798.450556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12798.467671] Lustre: Skipped 3 previous similar messages [12798.906758] Lustre: server umount lustre-MDT0000 complete [12802.579268] LustreError: 335735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786959764 with bad export cookie 15416949566723834166 [12802.583259] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12802.594843] LustreError: 335735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12802.943222] Lustre: server umount lustre-MDT0001 complete [12817.428842] Lustre: server umount lustre-OST0000 complete [12820.450342] Lustre: 335017:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786959765/real 1786959765] req@ffff91d2d24d8e00 x1873763010411648/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786959781 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12820.476407] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12822.184597] Lustre: server umount lustre-OST0001 complete [12845.657957] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [12857.264245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12857.914237] LustreError: 341036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12858.046794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12863.167725] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12863.463107] LustreError: 341037:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12863.471387] LustreError: 341037:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12867.582092] LustreError: 341036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12872.269169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12872.755702] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12877.506856] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12880.837657] Lustre: 342176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12888.996501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12889.295236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12896.720523] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12897.450895] LustreError: 342533:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12902.599413] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12908.077563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12913.672054] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12916.132836] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [12925.166901] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12929.167995] Lustre: 344049:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12934.114626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12934.121883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12934.123338] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12934.134876] Lustre: Skipped 3 previous similar messages [12934.137302] Lustre: Skipped 2 previous similar messages [12939.244440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12939.254598] Lustre: Skipped 4 previous similar messages [12940.572496] Lustre: server umount lustre-MDT0000 complete [12944.374634] LustreError: 341037:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12944.415449] LustreError: 341037:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [12945.377479] LustreError: 342178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786959906 with bad export cookie 15416949566723835867 [12945.387087] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12945.398871] LustreError: 342178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12946.110459] Lustre: server umount lustre-MDT0001 complete [12961.605643] Lustre: server umount lustre-OST0000 complete [12965.868436] Lustre: 335020:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786959911/real 1786959911] req@ffff91d2c10ac700 x1873763010462080/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786959927 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12965.902827] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12966.625791] Lustre: server umount lustre-OST0001 complete [12971.068294] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12972.677293] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12974.579565] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:45:34 (1786959934) [13000.750375] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [13011.205937] loop0: detected capacity change from 0 to 1099511627776 [13012.097463] SGI XFS with ACLs, security attributes, quota, no debug enabled [13012.107384] *** VALIDATE xfs *** [13012.140367] XFS (loop0): Mounting V5 Filesystem [13012.214108] XFS (loop0): Ending clean mount [13013.940280] loop1: detected capacity change from 0 to 18691697672192 [13139.128413] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [13151.937529] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [13192.299664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13192.618622] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13192.674472] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13192.818493] Lustre: lustre-MDT0000: new disk, initializing [13192.926317] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13192.934603] Lustre: Skipped 1 previous similar message [13192.958213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13199.081411] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13213.575874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13213.768976] Lustre: 349190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13213.857106] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13213.860753] Lustre: Skipped 1 previous similar message [13213.936965] Lustre: lustre-MDT0001: new disk, initializing [13214.040936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13214.053237] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13218.916990] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13229.393384] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13236.557971] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13244.740491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13245.116171] Lustre: lustre-OST0000: new disk, initializing [13245.121094] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13245.140744] Lustre: 351159:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13247.141850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13247.156612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13247.333366] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13252.800526] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13263.963112] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13268.985133] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13269.319531] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13274.414822] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13274.708645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13278.184301] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13278.204139] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13278.241249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13283.318924] LustreError: 351144:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13283.538758] Lustre: server umount lustre-OST0000 complete [13288.418021] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13288.433864] Lustre: Skipped 1 previous similar message [13288.438868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13288.451623] Lustre: Skipped 2 previous similar messages [13294.379991] Lustre: server umount lustre-MDT0000 complete [13299.202049] LustreError: 349984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786960260 with bad export cookie 15416949566723837421 [13299.204220] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13299.216573] LustreError: 349984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13299.609193] Lustre: server umount lustre-MDT0001 complete [13309.300883] XFS (loop0): Unmounting Filesystem [13314.614282] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 05:51:13 (1786960273) [13323.608912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13324.104972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13324.118783] Lustre: Skipped 2 previous similar messages [13328.705664] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13338.096374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13344.175712] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13352.869948] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13358.786417] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13366.232637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13373.265427] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13382.021584] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13394.405806] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13394.406414] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13394.423828] Lustre: Skipped 1 previous similar message [13394.426700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13394.441186] LustreError: Skipped 1 previous similar message [13394.453314] Lustre: Skipped 3 previous similar messages [13400.293424] Lustre: server umount lustre-OST0000 complete [13405.154641] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13411.411857] Lustre: server umount lustre-MDT0000 complete [13414.892982] LustreError: 353718:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13414.933972] LustreError: 353718:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [13415.558247] LustreError: 353698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786960377 with bad export cookie 15416949566723838303 [13415.571338] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13415.573225] LustreError: 353698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13415.992482] Lustre: server umount lustre-MDT0001 complete [13429.874662] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [13432.969530] Key type lgssc unregistered [13433.262260] LNet: 357350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13433.278444] LNetError: 357350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13434.358246] LNet: Removed LNI 192.168.204.118@tcp [13435.501186] Key type .llcrypt unregistered [13435.503851] Key type ._llcrypt unregistered [13466.549390] Key type ._llcrypt registered [13466.551553] Key type .llcrypt registered [13466.676586] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [13467.557561] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13467.660078] alg: No test for adler32 (adler32-zlib) [13468.782490] Lustre: Lustre: Build Version: 2.17.57_1_g72a0e6d [13469.096373] LNet: Added LNI 192.168.204.118@tcp [8/256/0/180] [13470.848195] Key type lgssc registered [13472.164822] Lustre: Echo OBD driver; http://www.lustre.org/ [13480.545295] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13480.566489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13482.000958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13486.583278] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13492.195676] LustreError: 358922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13492.208122] LustreError: 358922:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13495.935408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13496.296764] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13501.368781] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13509.998169] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13516.002401] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13523.129986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13523.388724] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13529.057935] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13536.429332] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13547.490357] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13547.500453] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13547.517874] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13548.590528] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.18@tcp (stopping) [13551.076670] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13551.079950] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13551.117720] Lustre: Skipped 1 previous similar message [13553.063372] Lustre: server umount lustre-MDT0001 complete [13553.661487] LustreError: 358918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13556.193878] LustreError: 360669:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13556.231948] LustreError: 360669:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13558.785451] LustreError: 358918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13563.900843] LustreError: 358917:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13563.918679] LustreError: 358917:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13564.684892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13564.898415] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13564.905302] Lustre: Found index 1 for lustre-MDT0001, updating log [13564.975226] Lustre: 358910:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13565.648751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13571.056163] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13571.552294] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13572.614502] LustreError: 362408:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13572.628681] LustreError: 362408:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13572.646718] LustreError: 358914:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13572.762345] LustreError: 362420:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.118@tcp: cfg command failed: rc = -17 [13572.775725] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13572.775725] [13584.397660] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.18@tcp (stopping) [13586.404528] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13586.412431] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13586.453669] Lustre: Skipped 1 previous similar message [13589.356640] Lustre: server umount lustre-MDT0001 complete [13589.490648] LustreError: 358919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13603.171840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13603.402657] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13603.409326] Lustre: Found index 1 for lustre-MDT0001, updating log [13603.472438] Lustre: 358911:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13603.896911] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13608.938843] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13609.084956] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13613.069918] LustreError: 363600:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13613.080337] LustreError: 363600:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13613.092482] LustreError: 358914:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13613.105840] LustreError: 358914:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13613.211177] LustreError: 363603:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13613.225380] LustreError: 363603:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13613.236544] LustreError: 363603:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.118@tcp: cfg command failed: rc = -17 [13613.248324] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.118@tcp [13613.248324] [13917.202383] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.18@tcp (stopping) [13917.219763] Lustre: Skipped 2 previous similar messages [13921.255665] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13921.259994] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13921.279212] Lustre: Skipped 1 previous similar message [13922.934583] Lustre: server umount lustre-MDT0001 complete [13926.371806] LustreError: 358922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13926.383397] LustreError: 358922:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [13935.718366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13935.883809] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13935.887812] Lustre: Found index 1 for lustre-MDT0001, updating log [13935.932211] Lustre: 358912:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13936.338069] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13941.454448] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [13941.748951] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13943.799547] LustreError: 372054:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13943.805839] LustreError: 372054:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13943.812944] LustreError: 358914:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13943.819303] LustreError: 358914:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13943.922766] LustreError: 372061:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13943.955649] LustreError: 372061:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.118@tcp: cfg command failed: rc = -114 [13943.973305] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13943.973305] [14264.298900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14264.303525] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14264.310247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14264.326527] Lustre: Skipped 2 previous similar messages [14264.341375] Lustre: Skipped 5 previous similar messages [14276.064091] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14276.170241] Lustre: server umount lustre-MDT0000 complete [14278.950501] LustreError: 362108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786961240 with bad export cookie 5371068048616019787 [14278.954313] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14278.960761] LustreError: 362108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14279.650826] LustreError: 358918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14279.657182] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14279.663821] LustreError: 358918:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [14283.685685] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14283.690383] Lustre: Skipped 7 previous similar messages [14285.361620] Lustre: server umount lustre-MDT0001 complete [14289.539664] Lustre: server umount lustre-OST0000 complete [14297.765981] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [14306.109573] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [14344.485663] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:08:24 (1786961304) [14352.059111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14352.262623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14352.321484] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14352.425248] Lustre: lustre-MDT0000: new disk, initializing [14352.533488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14352.546380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14356.872882] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [14368.034474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14368.169722] Lustre: 382918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14368.299271] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14368.309939] Lustre: Skipped 1 previous similar message [14368.387487] Lustre: lustre-MDT0001: new disk, initializing [14368.509114] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14368.543369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14368.562063] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14373.141544] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [14383.095756] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14389.309372] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14397.156850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14397.527480] Lustre: lustre-OST0000: new disk, initializing [14397.534653] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14397.546896] Lustre: 384891:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14397.704098] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14402.635691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14402.649291] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14402.757414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14404.222514] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_default_debug -1 all [14413.178957] Lustre: DEBUG MARKER: oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14421.998379] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14422.003911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14422.030259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14422.606438] Lustre: server umount lustre-MDT0000 complete [14423.524363] LustreError: 382931:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14423.554578] LustreError: 382931:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14426.473560] LustreError: 383728:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14426.476381] LustreError: 384890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786961388 with bad export cookie 5371068048616021740 [14426.482536] LustreError: MGC192.168.204.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14426.511535] LustreError: 384890:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14426.815739] Lustre: server umount lustre-MDT0001 complete [14441.150483] Lustre: server umount lustre-OST0000 complete [14469.237669] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing set_hostid [14480.258103] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing load_modules_local [14512.323524] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14109 sec ========= 06:11:11 (1786961471) [14514.016241] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:11:13 (1786961473) === [14517.588740] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:11:17 (1786961477) === [14562.320727] Lustre: DEBUG MARKER: oleg418-server.virtnet: executing unload_modules_local [14564.857962] Key type lgssc unregistered [14565.147614] LNet: 391259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14565.162087] LNetError: 391259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14565.180577] LNet: Removed LNI 192.168.204.118@tcp [14565.994449] Key type .llcrypt unregistered [14565.997661] Key type ._llcrypt unregistered