[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505218807 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.003324] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.006013] kvm-guest: setup PV IPIs [ 0.009320] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.010000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.010026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.011017] pid_max: default: 32768 minimum: 301 [ 0.012178] LSM: Security Framework initializing [ 0.013078] Yama: becoming mindful. [ 0.014047] SELinux: Initializing. [ 0.015090] *** VALIDATE selinux *** [ 0.029142] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.033447] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.034140] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.036092] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.037119] *** VALIDATE tmpfs *** [ 0.038442] *** VALIDATE proc *** [ 0.039223] *** VALIDATE cgroup *** [ 0.040009] *** VALIDATE cgroup2 *** [ 0.042056] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.043132] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.045008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.046028] Spectre V2 : User space: Vulnerable [ 0.047007] Speculative Store Bypass: Vulnerable [ 0.049725] debug: unmapping init [mem 0xffffffffab459000-0xffffffffab460fff] [ 0.051871] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.052599] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.053023] ... version: 2 [ 0.054011] ... bit width: 48 [ 0.055011] ... generic registers: 4 [ 0.056010] ... value mask: 0000ffffffffffff [ 0.057012] ... max period: 00007fffffffffff [ 0.058012] ... fixed-purpose events: 3 [ 0.059009] ... event mask: 000000070000000f [ 0.060283] rcu: Hierarchical SRCU implementation. [ 0.062383] smp: Bringing up secondary CPUs ... [ 0.063489] x86: Booting SMP configuration: [ 0.064022] .... node #0, CPUs: #1 #2 #3 [ 0.067491] smp: Brought up 1 node, 4 CPUs [ 0.069014] smpboot: Max logical packages: 1 [ 0.070019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.263016] node 0 deferred pages initialised in 189ms [ 0.267113] devtmpfs: initialized [ 0.268158] x86/mm: Memory block size: 128MB [ 0.271793] gcov: version magic: 0x41383552 [ 0.274399] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.278109] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.280395] pinctrl core: initialized pinctrl subsystem [ 0.283251] [ 0.283988] ************************************************************* [ 0.286021] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.289019] ** ** [ 0.292021] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.295020] ** ** [ 0.298019] ** This means that this kernel is built to expose internal ** [ 0.300016] ** IOMMU data structures, which may compromise security on ** [ 0.303019] ** your system. ** [ 0.306019] ** ** [ 0.309027] ** If you see this message and you are not debugging the ** [ 0.312020] ** kernel, report this immediately to your vendor! ** [ 0.314016] ** ** [ 0.317019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.319015] ************************************************************* [ 0.322808] NET: Registered protocol family 16 [ 0.324523] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.327080] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.330085] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.336093] cpuidle: using governor menu [ 0.337699] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.340589] PCI: Using configuration type 1 for base access [ 0.342138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.351071] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.353113] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.357081] cryptd: max_cpu_qlen set to 1000 [ 0.359227] ACPI: Added _OSI(Module Device) [ 0.361016] ACPI: Added _OSI(Processor Device) [ 0.363016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.365014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.371109] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.377313] ACPI: Interpreter enabled [ 0.379122] ACPI: PM: (supports S0 S3 S4 S5) [ 0.381026] ACPI: Using IOAPIC for interrupt routing [ 0.383242] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.387492] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.398267] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.403078] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.407033] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.411140] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.417778] acpiphp: Slot [2] registered [ 0.419169] acpiphp: Slot [5] registered [ 0.421177] acpiphp: Slot [6] registered [ 0.423235] acpiphp: Slot [7] registered [ 0.425248] acpiphp: Slot [8] registered [ 0.427183] acpiphp: Slot [9] registered [ 0.429183] acpiphp: Slot [10] registered [ 0.430153] acpiphp: Slot [3] registered [ 0.432141] acpiphp: Slot [4] registered [ 0.434159] acpiphp: Slot [11] registered [ 0.436121] acpiphp: Slot [12] registered [ 0.437142] acpiphp: Slot [13] registered [ 0.439179] acpiphp: Slot [14] registered [ 0.441149] acpiphp: Slot [15] registered [ 0.442138] acpiphp: Slot [16] registered [ 0.444148] acpiphp: Slot [17] registered [ 0.446155] acpiphp: Slot [18] registered [ 0.448165] acpiphp: Slot [19] registered [ 0.450142] acpiphp: Slot [20] registered [ 0.452115] acpiphp: Slot [21] registered [ 0.453129] acpiphp: Slot [22] registered [ 0.454084] acpiphp: Slot [23] registered [ 0.456097] acpiphp: Slot [24] registered [ 0.457144] acpiphp: Slot [25] registered [ 0.459107] acpiphp: Slot [26] registered [ 0.460107] acpiphp: Slot [27] registered [ 0.462129] acpiphp: Slot [28] registered [ 0.463074] acpiphp: Slot [29] registered [ 0.465095] acpiphp: Slot [30] registered [ 0.466111] acpiphp: Slot [31] registered [ 0.468064] PCI host bridge to bus 0000:00 [ 0.470021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.472025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.475026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.478045] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.481044] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.484044] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.486244] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.490236] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.493498] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.504032] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.508712] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.512035] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.515036] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.519032] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.522077] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.524745] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.528049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.531720] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.535016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.547015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.551734] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.557301] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.566026] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.577024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.593018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.602426] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.609015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.616027] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.633054] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.644185] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.654028] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.665030] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.685029] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.696044] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.704018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.709983] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.733017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.745381] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.753021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.760023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.776020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.789762] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.798029] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.810029] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.824020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.836897] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.840551] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.843539] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.846445] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.849313] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.854370] iommu: Default domain type: Passthrough [ 0.857779] SCSI subsystem initialized [ 0.859354] ACPI: bus type USB registered [ 0.861171] usbcore: registered new interface driver usbfs [ 0.866144] usbcore: registered new interface driver hub [ 0.868116] usbcore: registered new device driver usb [ 0.870246] pps_core: LinuxPPS API ver. 1 registered [ 0.872014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.876093] PTP clock support registered [ 0.879098] EDAC MC: Ver: 3.0.0 [ 0.881205] PCI: Using ACPI for IRQ routing [ 0.883971] NetLabel: Initializing [ 0.885014] NetLabel: domain hash size = 128 [ 0.887014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.889141] NetLabel: unlabeled traffic allowed by default [ 0.892083] vgaarb: loaded [ 0.893276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.895021] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.905342] clocksource: Switched to clocksource kvm-clock [ 1.016515] VFS: Disk quotas dquot_6.6.0 [ 1.018414] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.021075] *** VALIDATE ramfs *** [ 1.022648] *** VALIDATE hugetlbfs *** [ 1.023809] pnp: PnP ACPI init [ 1.026320] pnp: PnP ACPI: found 6 devices [ 1.043961] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.047859] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.050474] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.053447] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.057033] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.059797] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.063206] NET: Registered protocol family 2 [ 1.066410] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.072346] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.076578] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.082919] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.086724] TCP: Hash tables configured (established 65536 bind 65536) [ 1.089988] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.093088] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.095760] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.098741] NET: Registered protocol family 1 [ 1.102270] RPC: Registered named UNIX socket transport module. [ 1.104739] RPC: Registered udp transport module. [ 1.106600] RPC: Registered tcp transport module. [ 1.108880] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.111896] NET: Registered protocol family 44 [ 1.113885] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.116197] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.118289] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.120817] PCI: CLS 0 bytes, default 64 [ 1.122741] Unpacking initramfs... [ 2.506492] debug: unmapping init [mem 0xffff8a747cc54000-0xffff8a747ffbffff] [ 2.510713] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.513104] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.516503] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.019198] Initialise system trusted keyrings [ 3.021061] Key type blacklist registered [ 3.023378] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.032539] zbud: loaded [ 3.035780] *** VALIDATE nfs *** [ 3.037695] *** VALIDATE nfs4 *** [ 3.039193] pstore: using deflate compression [ 3.042622] Platform Keyring initialized [ 3.140354] NET: Registered protocol family 38 [ 3.142138] Key type asymmetric registered [ 3.143789] Asymmetric key parser 'x509' registered [ 3.146226] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.149528] io scheduler mq-deadline registered [ 3.151455] io scheduler kyber registered [ 3.153477] io scheduler bfq registered [ 3.155538] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.158086] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.161286] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.163975] ACPI: Power Button [PWRF] [ 3.168466] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.175111] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.190089] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.196594] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.222577] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.252936] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.283521] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.289838] Non-volatile memory driver v1.3 [ 3.292550] Linux agpgart interface v0.103 [ 3.323828] virtio_blk virtio1: [vda] 132376 512-byte logical blocks (67.8 MB/64.6 MiB) [ 3.328227] vda: detected capacity change from 0 to 67776512 [ 3.344639] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.347756] vdb: detected capacity change from 0 to 1073741824 [ 3.365408] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.368537] vdc: detected capacity change from 0 to 2621440000 [ 3.383787] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.386969] vdd: detected capacity change from 0 to 2621440000 [ 3.401751] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.404619] vde: detected capacity change from 0 to 4294967296 [ 3.422626] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.426211] vdf: detected capacity change from 0 to 4294967296 [ 3.434352] libphy: Fixed MDIO Bus: probed [ 3.440244] usbcore: registered new interface driver usbserial_generic [ 3.442188] usbserial: USB Serial support registered for generic [ 3.444340] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.448798] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.450503] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.452893] mousedev: PS/2 mouse device common for all mice [ 3.455890] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.457707] rtc_cmos 00:05: RTC can wake from S4 [ 3.462951] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.463384] rtc_cmos 00:05: registered as rtc0 [ 3.468480] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.471780] intel_pstate: CPU model not supported [ 3.474281] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.478542] hid: raw HID events driver (C) Jiri Kosina [ 3.480867] usbcore: registered new interface driver usbhid [ 3.482982] usbhid: USB HID core driver [ 3.484781] drop_monitor: Initializing network drop monitor service [ 3.487499] Initializing XFRM netlink socket [ 3.489618] NET: Registered protocol family 10 [ 3.492908] Segment Routing with IPv6 [ 3.494736] NET: Registered protocol family 17 [ 3.497274] mpls_gso: MPLS GSO support [ 3.503024] RAS: Correctable Errors collector initialized. [ 3.505310] AVX version of gcm_enc/dec engaged. [ 3.507308] AES CTR mode by8 optimization enabled [ 3.595298] sched_clock: Marking stable (3595038007, 0)->(4470913945, -875875938) [ 3.600268] registered taskstats version 1 [ 3.602489] Loading compiled-in X.509 certificates [ 3.604725] zswap: loaded using pool lzo/zbud [ 3.638386] Key type big_key registered [ 3.651972] Key type encrypted registered [ 3.653640] ima: No TPM chip found, activating TPM-bypass! [ 3.655573] ima: Allocated hash algorithm: sha1 [ 3.657229] ima: No architecture policies found [ 3.658988] evm: Initialising EVM extended attributes: [ 3.661607] evm: security.selinux [ 3.663117] evm: security.ima [ 3.664509] evm: security.capability [ 3.665979] evm: HMAC attrs: 0x1 [ 3.668489] rtc_cmos 00:05: setting system clock to 2026-03-30 12:39:41 UTC (1774874381) [ 3.674723] debug: unmapping init [mem 0xffffffffac403000-0xffffffffac5fffff] [ 3.678399] debug: unmapping init [mem 0xffffffffab182000-0xffffffffab458fff] [ 3.687102] Write protecting the kernel read-only data: 28672k [ 3.690943] debug: unmapping init [mem 0xffffffffa9803000-0xffffffffa99fffff] [ 3.693923] debug: unmapping init [mem 0xffffffffaa114000-0xffffffffaa1fffff] [ 3.730457] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.741397] systemd[1]: Detected virtualization kvm. [ 3.742931] systemd[1]: Detected architecture x86-64. [ 3.744272] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.771731] systemd[1]: No hostname configured. [ 3.773893] systemd[1]: Set hostname to . [ 3.776340] random: systemd: uninitialized urandom read (16 bytes read) [ 3.779247] systemd[1]: Initializing machine ID from random generator. [ 3.827348] random: ln: uninitialized urandom read (6 bytes read) [ 3.927764] random: systemd: uninitialized urandom read (16 bytes read) [ 3.930977] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.938417] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.948205] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local File Systems. Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.580068] device-mapper: uevent: version 1.0.3 [ 4.582572] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ 5.368367] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ 5.391714] random: fast init done [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.471252] scsi host0: ata_piix [ 5.495307] scsi host1: ata_piix [ 5.497286] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.499473] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.317363] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.172827] random: crng init done [ 10.174429] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.752695] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.962369] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.234428] SELinux: Disabled at runtime. [ 12.302130] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.311593] systemd[1]: Detected virtualization kvm. [ 12.313484] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.894553] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.898777] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.904480] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.909186] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.914299] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.922674] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.932993] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. Starting udev Coldplug all Devices... [ 13.118437] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.405399] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.800286] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.893763] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.010778] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.024824] EDAC sbridge: Ver: 1.1.2 [ 15.708547] Key type dns_resolver registered [ 16.042558] NFS: Registering the id_resolver key type [ 16.044718] Key type id_resolver registered [ 16.046452] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg415-server login: [ 42.906723] libcfs: loading out-of-tree module taints kernel. [ 42.923905] Key type ._llcrypt registered [ 42.925850] Key type .llcrypt registered [ 42.988657] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 54.367229] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 56.376322] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 56.412410] alg: No test for adler32 (adler32-zlib) [ 58.048449] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 59.381118] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 61.224144] Key type lgssc registered [ 63.709952] Lustre: Echo OBD driver; http://www.lustre.org/ [ 67.107012] hrtimer: interrupt took 2478253 ns [ 87.651957] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 132.807827] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 147.642399] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 147.696340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 148.983598] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 149.019977] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 149.102386] Lustre: lustre-MDT0000: new disk, initializing [ 149.183718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 149.208709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.069341] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.879590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.985874] Lustre: 6480:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 173.015964] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 173.023823] Lustre: Skipped 1 previous similar message [ 173.111770] Lustre: lustre-MDT0001: new disk, initializing [ 173.167030] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 173.194991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 173.204412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 178.432484] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.137943] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 196.034901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 196.396759] Lustre: lustre-OST0000: new disk, initializing [ 196.403265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.460182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 203.120657] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 206.390224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 206.400219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 206.509593] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 218.375438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 218.553164] Lustre: lustre-OST0001: new disk, initializing [ 218.566921] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 218.628916] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 225.657921] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 227.416158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 227.434531] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 227.512855] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 238.382771] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 247.072737] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 258.358977] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing check_logdir /tmp/testlogs/ [ 263.114555] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing yml_node [ 267.806921] Lustre: DEBUG MARKER: Client: 2.17.51.23 [ 270.616876] Lustre: DEBUG MARKER: MDS: 2.17.51.23 [ 273.699952] Lustre: DEBUG MARKER: OSS: 2.17.51.23 [ 275.494957] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Mar 30 08:44:11 EDT 2026 [ 296.876694] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 298.610629] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 316.904972] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 316.922097] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.956551] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 319.974323] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.977703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 320.007556] Lustre: Skipped 2 previous similar messages [ 320.028189] Lustre: Skipped 3 previous similar messages [ 322.088793] Lustre: server umount lustre-MDT0000 complete [ 327.222579] LustreError: 8384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774874705 with bad export cookie 3675250222787107751 [ 327.225756] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 327.243235] LustreError: 8384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 330.210728] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 330.211157] LustreError: 6488:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.211225] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 330.219345] Lustre: Skipped 1 previous similar message [ 330.230645] LustreError: 6488:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 6 previous similar messages [ 332.258261] LustreError: 6487:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 332.259806] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 332.289841] LustreError: 6487:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 332.291431] Lustre: Skipped 2 previous similar messages [ 333.726662] Lustre: server umount lustre-MDT0001 complete [ 340.917554] Lustre: server umount lustre-OST0000 complete [ 346.376594] Lustre: server umount lustre-OST0001 complete [ 355.433846] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 364.898272] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 403.238526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 403.497636] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 403.536579] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 403.631418] Lustre: lustre-MDT0000: new disk, initializing [ 403.735103] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 403.762940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 407.967600] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 419.908380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 420.094416] Lustre: 14960:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 420.168502] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 420.180457] Lustre: Skipped 1 previous similar message [ 420.310115] Lustre: lustre-MDT0001: new disk, initializing [ 420.491796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 420.567084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 420.579831] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 425.935623] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 435.080920] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 439.896364] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 448.609281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 448.995298] Lustre: lustre-OST0000: new disk, initializing [ 449.004678] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 449.171542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 450.791177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 450.819257] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 451.055464] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 458.020395] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 468.599688] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 474.686438] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 474.933449] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 479.499178] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 479.791982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 481.761069] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 481.762323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 481.770513] LustreError: Skipped 1 previous similar message [ 481.777039] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 481.783081] Lustre: Skipped 1 previous similar message [ 481.805098] Lustre: Skipped 1 previous similar message [ 487.895765] Lustre: server umount lustre-OST0000 complete [ 492.012809] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.028935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 492.033432] Lustre: Skipped 1 previous similar message [ 492.035179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 492.041908] Lustre: Skipped 3 previous similar messages [ 506.336122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 506.796385] Lustre: server umount lustre-MDT0000 complete [ 507.367267] LustreError: 14971:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 507.397153] LustreError: 14971:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 511.463574] LustreError: 15819:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.481097] LustreError: 15819:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 512.081403] LustreError: 14952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774874889 with bad export cookie 3675250222787109284 [ 512.090638] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 512.095949] LustreError: 14952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 512.534954] Lustre: server umount lustre-MDT0001 complete [ 518.616495] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:48:14 (1774874894) [ 526.536733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 527.160180] LustreError: 18824:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.308618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 531.825085] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 537.570460] LustreError: 18825:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.586071] LustreError: 18825:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 540.898026] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 547.963290] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 556.320940] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 562.095929] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 571.292351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 579.139960] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 586.029554] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 589.765143] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 597.474851] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 597.494143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 597.519211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 597.528207] Lustre: Skipped 4 previous similar messages [ 602.949135] Lustre: server umount lustre-MDT0000 complete [ 606.691076] LustreError: 18820:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 606.723301] LustreError: 18820:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 607.136997] LustreError: 18804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774874984 with bad export cookie 3675250222787110166 [ 607.139020] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 607.153836] LustreError: 18804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 607.487513] Lustre: server umount lustre-MDT0001 complete [ 622.247914] Lustre: server umount lustre-OST0000 complete [ 644.464489] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 654.682826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 655.080928] LustreError: 22854:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 655.137199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 655.146404] Lustre: Skipped 2 previous similar messages [ 659.544858] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 668.326313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 673.685502] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 676.833155] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 684.630948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 690.733154] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 699.293791] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 699.521927] Lustre: lustre-OST0001: new disk, initializing [ 699.528278] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 700.647590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 700.665536] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 700.782391] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 706.244850] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 717.898382] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 721.609782] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 740.326844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 740.337122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 740.348780] Lustre: Skipped 2 previous similar messages [ 740.359941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 740.365545] Lustre: Skipped 3 previous similar messages [ 746.150121] Lustre: server umount lustre-MDT0000 complete [ 746.979458] LustreError: 22850:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 747.011168] LustreError: 22850:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 750.522821] LustreError: 25125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875128 with bad export cookie 3675250222787111013 [ 750.531092] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 750.540194] LustreError: 25125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 750.893940] Lustre: server umount lustre-MDT0001 complete [ 764.876780] Lustre: server umount lustre-OST0000 complete [ 767.968420] Lustre: 3629:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774875129/real 1774875129] req@ffff8a73c79dd180 x1861090737904896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1774875145 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 769.318745] Lustre: server umount lustre-OST0001 complete [ 787.580507] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 799.848335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 800.510423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 800.516192] Lustre: Skipped 3 previous similar messages [ 805.215794] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 814.880487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.554368] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 823.991186] Lustre: 29226:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 832.168224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 839.903822] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 848.805426] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 855.857716] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 865.465360] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 869.996362] Lustre: 31077:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 881.633294] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 881.646938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 881.669657] Lustre: Skipped 4 previous similar messages [ 881.682198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 881.697101] Lustre: Skipped 4 previous similar messages [ 888.015455] Lustre: server umount lustre-MDT0000 complete [ 890.339259] LustreError: 28111:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 890.352809] LustreError: 28111:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 10 previous similar messages [ 892.791852] LustreError: 28846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875270 with bad export cookie 3675250222787112308 [ 892.797278] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 892.802136] LustreError: 28846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 893.203917] Lustre: server umount lustre-MDT0001 complete [ 907.877028] Lustre: server umount lustre-OST0000 complete [ 921.909557] Lustre: server umount lustre-OST0001 complete [ 932.170908] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:55:08 (1774875308) [ 967.572749] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 980.431414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 986.293842] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 995.415522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1000.681418] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1004.053918] Lustre: 34984:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1011.401622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1017.692697] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1025.318247] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1031.500930] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1039.186785] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1043.337108] Lustre: 36837:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1057.262932] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1057.263496] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1057.280570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1057.292220] Lustre: Skipped 6 previous similar messages [ 1057.318808] Lustre: Skipped 7 previous similar messages [ 1060.777259] Lustre: server umount lustre-MDT0000 complete [ 1064.756463] LustreError: 36839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875442 with bad export cookie 3675250222787113414 [ 1064.759620] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1064.773541] LustreError: 36839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1065.188161] Lustre: server umount lustre-MDT0001 complete [ 1079.298841] Lustre: server umount lustre-OST0000 complete [ 1093.411508] Lustre: server umount lustre-OST0001 complete [ 1102.136892] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:57:58 (1774875478) [ 1119.631444] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1129.385432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1129.885691] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1129.899859] Lustre: Skipped 7 previous similar messages [ 1134.331847] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1143.459864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1149.506325] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1152.811867] Lustre: 40080:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1160.195022] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1167.209238] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1168.873065] LustreError: 40432:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1168.908136] LustreError: 40432:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 11 previous similar messages [ 1174.890038] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1181.331295] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1188.048733] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1191.266526] Lustre: 41932:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1197.173167] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1197.181207] Lustre: Skipped 1 previous similar message [ 1200.098459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1200.106714] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1206.323877] Lustre: server umount lustre-MDT0000 complete [ 1209.490756] LustreError: 38945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875587 with bad export cookie 3675250222787114513 [ 1209.492768] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1209.506750] LustreError: 38945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1215.883348] Lustre: server umount lustre-MDT0001 complete [ 1224.890719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1235.876216] LustreError: 3628:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8a7500948e00 x1861090738068480/t0(0) o250->MGC192.168.204.115@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1240.676439] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1244.649521] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1250.812307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1254.522092] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1254.534899] Lustre: Skipped 1 previous similar message [ 1256.996234] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1264.153801] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1266.047542] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1270.050626] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1271.880749] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1276.900420] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1283.250901] Lustre: server umount lustre-MDT0000 complete [ 1287.387869] LustreError: 39699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875665 with bad export cookie 3675250222787115682 [ 1287.392031] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1287.409499] LustreError: 39699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1287.753609] Lustre: server umount lustre-MDT0001 complete [ 1302.157731] Lustre: server umount lustre-OST0000 complete [ 1316.943600] Lustre: server umount lustre-OST0001 complete [ 1324.822453] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:01:41 (1774875701) [ 1339.330461] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1349.075567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1354.516447] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1364.540299] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1370.530743] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1374.141349] Lustre: 47440:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1383.091772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1390.744763] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1401.488419] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.430606] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1418.883927] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1428.874942] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1441.762309] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1441.777829] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1441.791485] Lustre: Skipped 16 previous similar messages [ 1441.965182] Lustre: server umount lustre-MDT0000 complete [ 1445.972767] LustreError: 49297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774875823 with bad export cookie 3675250222787116501 [ 1445.976289] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1445.981852] LustreError: 49297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1446.473689] Lustre: server umount lustre-MDT0001 complete [ 1460.840723] Lustre: server umount lustre-OST0000 complete [ 1464.801056] Lustre: 3631:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774875826/real 1774875826] req@ffff8a74c425d500 x1861090738168832/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1774875842 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1464.825597] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1464.841780] Lustre: Skipped 13 previous similar messages [ 1465.207480] Lustre: server umount lustre-OST0001 complete [ 1475.315381] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:04:11 (1774875851) [ 1477.871878] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1479.931656] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:04:15 (1774875855) [ 1491.509747] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 1500.572666] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1523.181930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1523.481990] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1523.554776] Lustre: lustre-MDT0000: new disk, initializing [ 1523.656550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1527.284903] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1534.350748] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1542.662488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1542.969159] Lustre: lustre-OST0000: new disk, initializing [ 1542.972878] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1542.988465] Lustre: Skipped 1 previous similar message [ 1544.229947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1544.253186] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1544.293528] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1550.614854] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1560.511320] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1562.508618] Lustre: DEBUG MARKER: create test files [ 1598.585763] Lustre: server umount lustre-OST0000 complete [ 1602.575731] Lustre: server umount lustre-MDT0000 complete [ 1613.714198] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 1616.553171] Key type lgssc unregistered [ 1616.919229] LNet: 54843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1616.927557] LNetError: 54843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1616.946059] LNet: Removed LNI 192.168.204.115@tcp [ 1617.959342] Key type .llcrypt unregistered [ 1617.962665] Key type ._llcrypt unregistered [ 1647.679443] Key type ._llcrypt registered [ 1647.681411] Key type .llcrypt registered [ 1647.790150] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1648.873539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1648.966117] alg: No test for adler32 (adler32-zlib) [ 1650.082169] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 1650.375401] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 1652.092227] Key type lgssc registered [ 1653.161303] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1660.805255] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1660.828563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1662.507424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1667.076293] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1673.085911] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1680.002383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1680.457713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1687.408134] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1687.624465] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1687.741133] Lustre: 57453:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1687.751672] Lustre: 57453:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1687.765566] Lustre: 57453:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1687.778031] Lustre: 57453:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1687.799281] Lustre: 57453:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1687.808071] Lustre: 57453:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1693.409300] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1698.390585] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1703.906593] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1712.904866] Lustre: server umount lustre-OST0000 complete [ 1726.741196] Lustre: server umount lustre-MDT0000 complete [ 1737.322127] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 1739.591257] Key type lgssc unregistered [ 1739.860620] LNet: 58933:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1739.869890] LNetError: 58933:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1739.884690] LNet: Removed LNI 192.168.204.115@tcp [ 1740.790491] Key type .llcrypt unregistered [ 1740.793667] Key type ._llcrypt unregistered [ 1774.153125] Key type ._llcrypt registered [ 1774.158680] Key type .llcrypt registered [ 1774.270166] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1775.178287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1775.197395] alg: No test for adler32 (adler32-zlib) [ 1776.164085] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 1776.423141] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 1778.124647] Key type lgssc registered [ 1779.284736] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1786.294450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1786.326007] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1788.032076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1792.524907] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1797.577300] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1804.934825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1805.302303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1811.400447] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1812.470772] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1812.593318] Lustre: 61639:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1812.601473] Lustre: 61639:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1812.609520] Lustre: 61639:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1812.617290] Lustre: 61639:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1812.627224] Lustre: 61639:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1812.634442] Lustre: 61639:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1817.629844] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1823.864605] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1832.932088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1832.946922] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1838.049280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1838.391739] Lustre: server umount lustre-OST0000 complete [ 1842.871676] Lustre: server umount lustre-MDT0000 complete [ 1856.691843] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 1860.280666] Key type lgssc unregistered [ 1860.649714] LNet: 63068:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1860.661139] LNetError: 63068:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1860.687062] LNet: Removed LNI 192.168.204.115@tcp [ 1861.569499] Key type .llcrypt unregistered [ 1861.573392] Key type ._llcrypt unregistered [ 1881.476562] Key type ._llcrypt registered [ 1881.481256] Key type .llcrypt registered [ 1881.638916] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 1897.075826] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 1898.148447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1898.165444] alg: No test for adler32 (adler32-zlib) [ 1899.194982] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 1899.422921] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 1901.096254] Key type lgssc registered [ 1902.200428] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1937.417168] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1937.451946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1938.671660] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1938.746901] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1938.841813] Lustre: lustre-MDT0000: new disk, initializing [ 1938.918799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1938.948219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1943.312408] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1954.026205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1954.132936] Lustre: 66196:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1954.253415] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1954.261951] Lustre: Skipped 1 previous similar message [ 1954.361643] Lustre: lustre-MDT0001: new disk, initializing [ 1954.478832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1954.544459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1954.572066] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1959.281837] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1967.569728] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1971.551756] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1979.313082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1979.683351] Lustre: lustre-OST0000: new disk, initializing [ 1979.692283] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1979.881738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1986.764766] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 1989.754392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1989.766017] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1989.845778] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1995.037113] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1999.133811] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1999.328533] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2003.138833] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2003.319602] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2005.475763] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2005.488329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2005.506733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2005.511064] Lustre: Skipped 1 previous similar message [ 2010.603395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2010.616825] Lustre: Skipped 1 previous similar message [ 2015.718367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2015.725761] Lustre: Skipped 1 previous similar message [ 2019.296228] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2019.486199] Lustre: server umount lustre-OST0000 complete [ 2025.954980] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2025.957879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2025.966713] Lustre: Skipped 2 previous similar messages [ 2025.976944] Lustre: Skipped 1 previous similar message [ 2029.904202] Lustre: server umount lustre-MDT0000 complete [ 2031.076898] LustreError: 66206:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2031.092562] LustreError: 66206:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 2033.668181] LustreError: 66186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774876411 with bad export cookie 6791174527470189301 [ 2033.673096] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2033.677562] LustreError: 66186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2034.015972] Lustre: server umount lustre-MDT0001 complete [ 2043.827854] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:13:39 (1774876419) [ 2056.302928] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 2095.904352] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 2106.434268] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 2152.201308] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2152.488391] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2152.547754] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2152.630277] Lustre: lustre-MDT0000: new disk, initializing [ 2152.737545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2152.776874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2157.366874] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2168.339734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2168.507863] Lustre: 73759:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2168.612043] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2168.618934] Lustre: Skipped 1 previous similar message [ 2168.775789] Lustre: lustre-MDT0001: new disk, initializing [ 2168.932854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2169.003683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2169.023367] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2173.720747] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2181.288936] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2185.509916] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2193.086936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2193.472917] Lustre: lustre-OST0000: new disk, initializing [ 2193.481261] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2193.604461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2199.641000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2199.668671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2199.830978] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2200.938646] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2209.952877] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2215.244375] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2215.540560] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2221.688612] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2221.983949] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2225.639573] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2225.640143] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2225.646534] LustreError: Skipped 2 previous similar messages [ 2225.651399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2225.669354] Lustre: Skipped 1 previous similar message [ 2230.439284] Lustre: server umount lustre-OST0000 complete [ 2235.361552] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2235.386413] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2235.431840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2235.441640] Lustre: Skipped 1 previous similar message [ 2240.994895] LustreError: 73772:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2241.021508] LustreError: 73772:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 2241.101673] Lustre: server umount lustre-MDT0000 complete [ 2245.207635] LustreError: 75500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774876623 with bad export cookie 6791174527470190176 [ 2245.214941] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2245.216996] LustreError: 75500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2245.724819] Lustre: server umount lustre-MDT0001 complete [ 2255.771993] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:17:11 (1774876631) [ 2264.382637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2265.083137] LustreError: 77767:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2265.161784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2270.181035] LustreError: 77768:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2270.968662] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2274.282428] LustreError: 77767:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2281.929646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2287.062339] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2293.579445] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2298.025767] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2305.618916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2306.144295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2306.149836] Lustre: Skipped 1 previous similar message [ 2313.013274] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2319.833696] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2323.913588] Lustre: 80103:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting 50def04d-a940-42fd-8ab5-6f47b0a7420e at adminstrative request [ 2324.973185] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2324.997956] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2334.183813] Lustre: MGS: Client 50def04d-a940-42fd-8ab5-6f47b0a7420e (at 0@lo) reconnecting [ 2334.208894] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5e3f1940658ac1d9 to 0x5e3f1940658ac479 [ 2334.238188] Lustre: MGC192.168.204.115@tcp: Connection restored to 0@lo (at 0@lo) [ 2361.739422] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2361.980954] Lustre: lustre-OST0001: new disk, initializing [ 2361.987960] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2369.487785] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2370.826436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2370.848439] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2371.015793] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2377.730185] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2381.283583] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2381.290119] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2381.296813] LustreError: Skipped 1 previous similar message [ 2381.297715] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2381.297722] Lustre: Skipped 1 previous similar message [ 2381.317054] Lustre: Skipped 2 previous similar messages [ 2395.106257] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2395.486302] Lustre: server umount lustre-OST0001 complete [ 2396.642807] LustreError: 79360:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2396.671312] LustreError: 79360:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 2401.764293] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2401.771582] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2401.783665] Lustre: Skipped 1 previous similar message [ 2415.014253] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2415.021925] Lustre: Skipped 11 previous similar messages [ 2415.023209] LustreError: 79536:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2415.047967] LustreError: 79536:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 7 previous similar messages [ 2415.072212] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2415.306948] Lustre: server umount lustre-OST0000 complete [ 2420.198872] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2420.221054] Lustre: Skipped 1 previous similar message [ 2425.904852] Lustre: server umount lustre-MDT0000 complete [ 2430.169055] LustreError: 80501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774876808 with bad export cookie 6791174527470191737 [ 2430.178076] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2430.195156] LustreError: 80501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2430.635249] Lustre: server umount lustre-MDT0001 complete [ 2439.793767] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:20:15 (1774876815) [ 2441.517969] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2443.310238] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:20:19 (1774876819) [ 2445.330559] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2447.458551] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:20:23 (1774876823) [ 2449.656603] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2451.678201] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:20:27 (1774876827) [ 2453.611736] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2456.031506] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:20:31 (1774876831) [ 2458.168752] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2466.332689] loop: module loaded [ 2466.364978] loop0: detected capacity change from 0 to 4294967296 [ 2466.456099] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2468.600677] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2473.467992] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2477.360496] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2507.795614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2507.847263] LustreError: 84500:0:(obd_config.c:843:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2507.868612] LustreError: 84500:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2507.873855] LustreError: 84500:0:(tgt_mount.c:2533:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2507.890264] LustreError: 84500:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2529.371220] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 2539.920605] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 2581.831398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2582.278939] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2582.366238] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2582.524244] Lustre: lustre-MDT0000: new disk, initializing [ 2582.764372] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2582.777761] Lustre: Skipped 1 previous similar message [ 2582.823986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2589.155471] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2600.516304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2600.650298] Lustre: 87334:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2600.995095] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2605.391593] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2613.848340] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2618.298828] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2625.606120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2632.984345] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2634.303652] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2634.447974] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2641.217053] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2646.754961] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2646.990810] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2652.160962] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2652.527524] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2655.213170] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2655.218317] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2655.267942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2655.274124] Lustre: Skipped 6 previous similar messages [ 2661.226861] Lustre: server umount lustre-OST0000 complete [ 2672.922463] Lustre: server umount lustre-MDT0000 complete [ 2675.682135] LustreError: 88439:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2675.718642] LustreError: 88439:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 2678.471064] LustreError: 87327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774877056 with bad export cookie 6791174527470192353 [ 2678.489689] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2679.172267] Lustre: server umount lustre-MDT0001 complete [ 2691.246353] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:24:26 (1774877066) [ 2700.389282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2708.296922] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2719.957947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2720.488239] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2720.496495] Lustre: Skipped 3 previous similar messages [ 2726.214716] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2733.799878] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2738.456765] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2746.181233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2753.390601] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2759.178298] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2763.919563] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2771.718665] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 7 sec [ 2776.178393] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2776.486522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2783.715529] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2783.880332] Lustre: lustre-OST0001: new disk, initializing [ 2783.884641] Lustre: Skipped 2 previous similar messages [ 2783.888979] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2783.896209] Lustre: Skipped 3 previous similar messages [ 2789.469270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2789.474489] Lustre: Skipped 2 previous similar messages [ 2789.489837] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2789.581569] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2790.092714] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2798.406100] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2804.005434] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2804.347658] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2809.385510] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2809.706661] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2831.703158] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2866.102197] Lustre: Failing over lustre-MDT0000 [ 2866.659772] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2866.679303] Lustre: Skipped 3 previous similar messages [ 2866.691279] LustreError: 91340:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2866.713792] LustreError: 91340:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 2866.854419] Lustre: server umount lustre-MDT0000 complete [ 2888.161893] Lustre: 64420:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774877249/real 1774877249] req@000000009cf304d3 x1861092667314304/t0(0) o400->MGC192.168.204.115@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1774877265 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2888.210579] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2893.765213] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2893.767584] LDISKFS-fs (dm-0): recovery complete [ 2893.779044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2898.725924] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2898.918622] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2898.926272] Lustre: 93761:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2902.898446] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 2904.080288] Lustre: 92055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2904.103660] Lustre: 92055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2904.114574] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2904.230789] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2904.568675] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2904.908680] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2904.920179] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 2905.240166] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2905.247773] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 2905.256373] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2905.265352] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2906.226827] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2906.245486] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 3 previous similar messages [ 2906.569627] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2906.576852] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 3 previous similar messages [ 2907.554700] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2907.579925] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2908.529156] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2908.542037] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 6 previous similar messages [ 2908.864207] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2908.877403] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 6 previous similar messages [ 2911.740925] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2911.754146] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2912.707937] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2912.715108] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 12 previous similar messages [ 2913.032178] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2913.038895] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 12 previous similar messages [ 2920.002185] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2920.033221] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 2921.009854] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2921.022545] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 25 previous similar messages [ 2921.346553] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2921.359306] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 25 previous similar messages [ 2936.025860] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2936.047395] Lustre: 97102:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 49 previous similar messages [ 2937.009358] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2937.025890] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 49 previous similar messages [ 2937.352123] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2937.367272] LustreError: 97102:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 49 previous similar messages [ 2958.241014] Lustre: 97102:0:(ldlm_lib.c:2376:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2958.248495] Lustre: 97102:0:(ldlm_lib.c:2386:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2958.259232] Lustre: 97102:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 8092234b-45a9-45dd-a90c-a7756b219857@192.168.204.15@tcp [ 2958.273414] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2958.279772] Lustre: 97102:0:(ldlm_lib.c:1896:abort_req_replay_queue()) @@@ aborted: req@000000003f40910b x1861092651370880/t0(8589934766) o35->8092234b-45a9-45dd-a90c-a7756b219857@192.168.204.15@tcp:696/0 lens 392/0 e 0 to 0 dl 1774877346 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2958.360399] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2958.372684] Lustre: lustre-MDT0000: Denying connection for new client 8092234b-45a9-45dd-a90c-a7756b219857 (at 192.168.204.15@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2958.409315] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2958.477062] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2958.486744] Lustre: Skipped 2 previous similar messages [ 2958.489788] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2958.547172] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:545) [ 2958.552273] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2974.699975] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2974.720610] LustreError: Skipped 2 previous similar messages [ 2974.734711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2974.745052] Lustre: Skipped 5 previous similar messages [ 2980.540620] Lustre: server umount lustre-OST0000 complete [ 2990.914796] Lustre: server umount lustre-OST0001 complete [ 2994.677428] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2994.695954] Lustre: Skipped 7 previous similar messages [ 3000.846969] Lustre: server umount lustre-MDT0000 complete [ 3004.389686] LustreError: 93761:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3004.389686] LustreError: 91343:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3004.389698] LustreError: 91343:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 46 previous similar messages [ 3004.411770] LustreError: 93761:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 46 previous similar messages [ 3004.979642] LustreError: 91325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774877382 with bad export cookie 6791174527470243565 [ 3004.992909] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3004.995897] LustreError: 91325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3005.013118] LustreError: 64418:0:(import.c:716:ptlrpc_connect_import_locked()) already connecting [ 3005.313263] Lustre: server umount lustre-MDT0001 complete [ 3014.919686] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:29:50 (1774877390) [ 3023.149545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3023.162860] Lustre: *** cfs_fail_loc=197, val=0*** [ 3038.759294] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3043.872965] LustreError: 99065:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.115@tcp: failed processing log, type 4: rc = -110 [ 3049.109753] Lustre: *** cfs_fail_loc=197, val=0*** [ 3049.114028] Lustre: Skipped 65 previous similar messages [ 3069.664266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3069.679408] Lustre: Skipped 3 previous similar messages [ 3076.652690] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3082.891650] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3085.067763] Lustre: server umount lustre-OST0000 complete [ 3093.685839] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:31:10 (1774877470) [ 3115.619261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3121.664800] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3129.953478] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3133.749287] LustreError: 100605:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3133.760841] LustreError: 100605:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3133.769289] LustreError: 100605:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3134.059457] Lustre: server umount lustre-MDT0000 complete [ 3169.600833] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 3180.576178] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 3221.889419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3222.124922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3222.150260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3222.284653] Lustre: lustre-MDT0000: new disk, initializing [ 3222.408030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3227.450942] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3239.903409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3240.054333] Lustre: 104635:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3240.434903] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3246.510829] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3256.518523] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3260.392857] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3267.137048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3273.452358] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3274.123809] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3282.565433] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3287.261571] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3287.571946] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3292.358941] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3292.563535] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3299.299225] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3299.321064] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3299.332652] Lustre: Skipped 1 previous similar message [ 3299.337464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3299.340849] LustreError: Skipped 3 previous similar messages [ 3299.369275] Lustre: Skipped 10 previous similar messages [ 3301.031361] Lustre: server umount lustre-OST0000 complete [ 3311.555591] Lustre: server umount lustre-MDT0000 complete [ 3314.657969] LustreError: 105706:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3314.684835] LustreError: 105706:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 3315.910951] LustreError: 104628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774877693 with bad export cookie 6791174527470254121 [ 3315.920734] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3315.923843] LustreError: 104628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3316.486585] Lustre: server umount lustre-MDT0001 complete [ 3326.540864] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:35:02 (1774877702) [ 3336.192664] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 3340.269689] Key type lgssc unregistered [ 3340.672929] LNet: 108715:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3340.682765] LNetError: 108715:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3340.708604] LNet: Removed LNI 192.168.204.115@tcp [ 3341.834356] Key type .llcrypt unregistered [ 3341.836206] Key type ._llcrypt unregistered [ 3369.515289] Key type ._llcrypt registered [ 3369.522868] Key type .llcrypt registered [ 3369.805812] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 3386.631677] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 3387.888160] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3388.008145] alg: No test for adler32 (adler32-zlib) [ 3389.122894] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 3389.407103] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 3391.169728] Key type lgssc registered [ 3392.469641] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3438.325499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3438.358669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3439.808502] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3439.839627] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3439.919367] Lustre: lustre-MDT0000: new disk, initializing [ 3439.983413] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3440.002411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3444.426851] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3455.604905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3455.925351] Lustre: lustre-OST0000: new disk, initializing [ 3455.938986] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3455.954836] Lustre: Skipped 1 previous similar message [ 3456.052677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3462.680283] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3464.859757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3464.868661] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3464.932039] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3473.875958] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3473.994798] Lustre: lustre-OST0001: new disk, initializing [ 3473.998946] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3474.071244] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3479.998790] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3480.121106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3480.128504] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3480.159641] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3490.275873] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3490.317170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3495.393762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3495.408174] Lustre: Skipped 2 previous similar messages [ 3496.195074] Lustre: server umount lustre-MDT0000 complete [ 3502.925306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3507.756168] LustreError: 112045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774877885 with bad export cookie 10123250451707830426 [ 3507.768997] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3507.906522] Lustre: server umount lustre-OST0000 complete [ 3512.602844] Lustre: server umount lustre-OST0001 complete [ 3523.188437] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 09:38:19 (1774877899) [ 3535.013359] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 3561.188322] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:38:57 (1774877937) [ 3602.384566] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 3615.099261] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 3654.385107] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3654.654906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3654.707358] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3654.786315] Lustre: lustre-MDT0000: new disk, initializing [ 3654.972213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3655.014564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3661.236320] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3672.509876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3672.673888] Lustre: 120372:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3672.739661] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3672.748303] Lustre: Skipped 1 previous similar message [ 3672.914231] Lustre: lustre-MDT0001: new disk, initializing [ 3673.072288] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3673.124542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3673.140271] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3677.661794] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3686.505844] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3690.857946] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3699.517855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3699.839457] Lustre: lustre-OST0000: new disk, initializing [ 3699.843796] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3699.940941] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3707.126811] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3709.038456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3709.052063] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3709.187746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3715.691142] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3720.422471] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3720.663371] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3724.794648] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3724.973088] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3729.900851] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3729.901563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3729.926971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3729.945150] Lustre: Skipped 2 previous similar messages [ 3729.964656] Lustre: Skipped 1 previous similar message [ 3733.440818] Lustre: server umount lustre-OST0000 complete [ 3739.619910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3739.638839] LustreError: Skipped 1 previous similar message [ 3739.643125] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3739.665836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3744.772600] Lustre: server umount lustre-MDT0000 complete [ 3749.471355] LustreError: 120364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774878127 with bad export cookie 10123250451707831651 [ 3749.480765] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3749.484964] LustreError: 120364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3750.069452] Lustre: server umount lustre-MDT0001 complete [ 3761.096932] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:42:16 (1774878136) [ 3768.959886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3769.461695] LustreError: 124382:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3769.488666] LustreError: 124382:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 3769.540878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3774.922878] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3774.945625] LustreError: 124383:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.067327] LustreError: 124382:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3784.661507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3790.452155] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3797.698433] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3802.629173] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3810.564765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3810.883044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3810.893277] Lustre: Skipped 1 previous similar message [ 3818.117739] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3825.119383] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3833.758493] Lustre: *** cfs_fail_loc=159, val=0*** [ 3850.302190] Lustre: lustre-MDT0000: Client 58f99f63-1e8a-4843-9653-e428d4ed7f26 (at 192.168.204.15@tcp) reconnecting [ 3852.398851] Lustre: *** cfs_fail_loc=159, val=0*** [ 3852.408426] Lustre: Skipped 5 previous similar messages [ 3868.734300] Lustre: lustre-MDT0000: Client 58f99f63-1e8a-4843-9653-e428d4ed7f26 (at 192.168.204.15@tcp) reconnecting [ 3886.566969] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3886.581556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3886.597858] Lustre: Skipped 1 previous similar message [ 3886.606658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3886.624784] Lustre: Skipped 2 previous similar messages [ 3890.676774] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3890.682729] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3892.803553] Lustre: server umount lustre-OST0000 complete [ 3897.825826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3897.841701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3897.849896] Lustre: Skipped 2 previous similar messages [ 3900.901589] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3900.918027] Lustre: Skipped 1 previous similar message [ 3903.493753] Lustre: server umount lustre-MDT0000 complete [ 3906.021193] LustreError: 124382:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3906.033192] LustreError: 124382:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 3908.318833] LustreError: 124364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774878286 with bad export cookie 10123250451707832533 [ 3908.321521] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3908.333169] LustreError: 124364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3908.694410] Lustre: server umount lustre-MDT0001 complete [ 3923.551385] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 3926.473985] Key type lgssc unregistered [ 3926.746573] LNet: 127808:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3926.753449] LNetError: 127808:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3926.776994] LNet: Removed LNI 192.168.204.115@tcp [ 3927.927581] Key type .llcrypt unregistered [ 3927.933045] Key type ._llcrypt unregistered [ 3959.573242] Key type ._llcrypt registered [ 3959.582191] Key type .llcrypt registered [ 3959.760183] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 3961.291750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3961.309190] alg: No test for adler32 (adler32-zlib) [ 3962.496958] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 3962.780556] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 3964.489296] Key type lgssc registered [ 3965.724820] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3974.580835] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3974.598305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3976.125799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3980.688386] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 3985.382766] LustreError: 129210:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3985.398035] LustreError: 129210:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 3989.648283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3990.095661] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3995.659888] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4005.210517] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4011.679666] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4019.550632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4020.186234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4021.242242] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4028.725331] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4036.648976] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4048.574142] Lustre: *** cfs_fail_loc=159, val=0*** [ 4064.813910] Lustre: lustre-MDT0000: Client 6064271f-4590-400d-b6a5-96a47b756984 (at 192.168.204.15@tcp) reconnecting [ 4067.922241] Lustre: *** cfs_fail_loc=159, val=0*** [ 4067.927893] Lustre: Skipped 1 previous similar message [ 4084.278096] Lustre: lustre-MDT0001: Client 6064271f-4590-400d-b6a5-96a47b756984 (at 192.168.204.15@tcp) reconnecting [ 4086.211776] Lustre: *** cfs_fail_loc=159, val=0*** [ 4086.217794] Lustre: Skipped 3 previous similar messages [ 4101.734631] Lustre: lustre-MDT0001: Client 6064271f-4590-400d-b6a5-96a47b756984 (at 192.168.204.15@tcp) reconnecting [ 4107.038996] Lustre: 131913:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4110.563787] Lustre: *** cfs_fail_loc=159, val=0*** [ 4110.565881] Lustre: Skipped 3 previous similar messages [ 4126.763927] Lustre: lustre-MDT0000: Client 9b91f433-63a0-46c9-9e33-e6686321e1c8 (at 192.168.204.15@tcp) reconnecting [ 4129.425828] Lustre: *** cfs_fail_loc=159, val=0*** [ 4129.431066] Lustre: Skipped 14 previous similar messages [ 4145.776079] Lustre: lustre-MDT0000: Client 9b91f433-63a0-46c9-9e33-e6686321e1c8 (at 192.168.204.15@tcp) reconnecting [ 4147.711421] Lustre: 132202:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4154.337611] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4154.342207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4154.362390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4157.410779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4157.420892] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4157.421067] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4157.428448] Lustre: Skipped 1 previous similar message [ 4160.377421] Lustre: server umount lustre-OST0000 complete [ 4167.311556] Lustre: server umount lustre-MDT0000 complete [ 4167.649772] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4167.659384] LustreError: 129209:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4167.670578] Lustre: Skipped 1 previous similar message [ 4171.695734] LustreError: 129190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774878549 with bad export cookie 1184114488480673870 [ 4171.697695] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4171.706808] LustreError: 129190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4172.064977] Lustre: server umount lustre-MDT0001 complete [ 4184.871839] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 4187.784971] Key type lgssc unregistered [ 4188.165988] LNet: 133239:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4188.177477] LNetError: 133239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4189.223176] LNet: Removed LNI 192.168.204.115@tcp [ 4190.259668] Key type .llcrypt unregistered [ 4190.263930] Key type ._llcrypt unregistered [ 4218.428751] Key type ._llcrypt registered [ 4218.435502] Key type .llcrypt registered [ 4218.647984] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 4220.091820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4220.413443] alg: No test for adler32 (adler32-zlib) [ 4221.790953] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 4222.215271] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 4223.968187] Key type lgssc registered [ 4225.529792] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4234.400732] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4234.419472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4236.173406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4240.723231] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4245.483582] LustreError: 134636:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4245.514241] LustreError: 134636:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 4250.599803] LustreError: 134637:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4251.378138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4251.873163] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4257.235925] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4263.677902] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4268.048990] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4275.390055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4275.851862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4276.945146] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4281.033295] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4282.007631] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4288.705181] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4295.184543] Lustre: 136884:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4299.327768] Lustre: 136932:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4300.930086] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4301.788636] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4305.879906] Lustre: 137080:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4306.872577] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4307.919203] Lustre: 137178:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4313.062519] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4313.069925] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4313.087021] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4317.155049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4317.165256] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4317.166704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4319.254380] Lustre: server umount lustre-OST0000 complete [ 4323.812084] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4323.819569] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4323.833797] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4323.839247] Lustre: Skipped 1 previous similar message [ 4327.394079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4327.398889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4329.808849] Lustre: server umount lustre-MDT0000 complete [ 4332.513526] LustreError: 134632:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4332.544840] LustreError: 134632:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4334.047886] LustreError: 136982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774878711 with bad export cookie 15446032804634067199 [ 4334.053274] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4334.057895] LustreError: 136982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4334.440729] Lustre: server umount lustre-MDT0001 complete [ 4345.492207] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 4348.508965] Key type lgssc unregistered [ 4348.804653] LNet: 138223:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4348.810767] LNetError: 138223:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4348.830738] LNet: Removed LNI 192.168.204.115@tcp [ 4349.617288] Key type .llcrypt unregistered [ 4349.620798] Key type ._llcrypt unregistered [ 4375.818775] Key type ._llcrypt registered [ 4375.821484] Key type .llcrypt registered [ 4375.934184] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 4377.253239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4377.509214] alg: No test for adler32 (adler32-zlib) [ 4378.537464] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 4378.866487] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 4380.520170] Key type lgssc registered [ 4381.763974] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4390.636778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4390.662644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4392.198758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4396.520697] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4401.634877] LustreError: 139619:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4401.657671] LustreError: 139619:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 4406.298378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4406.766733] LustreError: 139620:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4407.128592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4412.437859] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4420.538911] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4427.291700] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4438.462408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4439.164987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4446.680844] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4451.434338] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4453.214915] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4461.703802] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4470.748365] Lustre: *** cfs_fail_loc=159, val=0*** [ 4487.214037] Lustre: lustre-MDT0000: Client 384749b8-e217-429a-b4ed-5c27eda6b9c1 (at 192.168.204.15@tcp) reconnecting [ 4497.893976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4497.897845] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4497.908362] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4497.935161] Lustre: Skipped 1 previous similar message [ 4500.116457] Lustre: server umount lustre-OST0000 complete [ 4504.545486] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4504.558365] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4504.579217] Lustre: Skipped 1 previous similar message [ 4504.584179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4508.134594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4508.144816] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4510.410962] Lustre: server umount lustre-MDT0000 complete [ 4513.263659] LustreError: 141853:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4513.273037] LustreError: 141853:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4514.379162] LustreError: 139602:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774878892 with bad export cookie 5341260501704173581 [ 4514.381674] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4514.395530] LustreError: 139602:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4514.708318] Lustre: server umount lustre-MDT0001 complete [ 4529.038192] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 4532.358481] Key type lgssc unregistered [ 4532.724427] LNet: 142952:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4532.743069] LNetError: 142952:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4532.758645] LNet: Removed LNI 192.168.204.115@tcp [ 4533.501693] Key type .llcrypt unregistered [ 4533.507633] Key type ._llcrypt unregistered [ 4556.582363] Key type ._llcrypt registered [ 4556.586983] Key type .llcrypt registered [ 4556.703548] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 4557.449867] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4557.519113] alg: No test for adler32 (adler32-zlib) [ 4558.717616] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 4558.913750] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 4560.601406] Key type lgssc registered [ 4561.879202] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4570.577366] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4570.624653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.322489] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4577.072449] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4582.883287] LustreError: 144350:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4582.913242] LustreError: 144350:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 4586.096035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4586.564494] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4591.325907] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4597.434778] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4603.189151] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4611.119832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4611.921913] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4612.996602] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4618.181557] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4618.645811] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4624.887724] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4645.519531] Lustre: lustre-MDT0001: evicting 38443e93-3375-4699-bdf0-c7a1c24aa95e (at 192.168.204.15@tcp) by administrative request [ 4645.531756] Lustre: Skipped 1 previous similar message [ 4679.650772] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4679.662157] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4679.673194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4679.680339] Lustre: Skipped 1 previous similar message [ 4684.774801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4684.796153] Lustre: Skipped 2 previous similar messages [ 4689.377069] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4689.653283] Lustre: server umount lustre-OST0000 complete [ 4694.404707] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4694.410520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4694.423821] Lustre: Skipped 1 previous similar message [ 4694.463446] Lustre: Skipped 1 previous similar message [ 4699.617492] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4699.632104] Lustre: Skipped 1 previous similar message [ 4700.734406] Lustre: server umount lustre-MDT0000 complete [ 4704.741622] LustreError: 144349:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4704.766087] LustreError: 144349:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4705.405279] LustreError: 144329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774879083 with bad export cookie 1135429396705457819 [ 4705.416851] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4705.427447] LustreError: 144329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4705.958540] Lustre: server umount lustre-MDT0001 complete [ 4718.241983] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 4721.977486] Key type lgssc unregistered [ 4722.424141] LNet: 148077:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4722.438657] LNetError: 148077:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4723.504714] LNet: Removed LNI 192.168.204.115@tcp [ 4724.718646] Key type .llcrypt unregistered [ 4724.721223] Key type ._llcrypt unregistered [ 4811.775426] Key type ._llcrypt registered [ 4811.777907] Key type .llcrypt registered [ 4811.902608] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 4813.040124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4813.077900] alg: No test for adler32 (adler32-zlib) [ 4814.235617] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 4814.627290] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 4816.336193] Key type lgssc registered [ 4817.405448] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4826.081828] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4826.121744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4827.922316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4832.680623] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4837.347324] LustreError: 150556:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4837.365811] LustreError: 150556:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 4842.334105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4842.473912] LustreError: 150557:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4842.865144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4847.764289] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4853.770413] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4858.051755] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4865.129674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4865.727398] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4867.793625] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4872.885733] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4873.393749] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 4880.646830] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4893.669335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4893.677123] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.680383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4893.691808] Lustre: Skipped 1 previous similar message [ 4893.705783] Lustre: Skipped 1 previous similar message [ 4897.375406] Lustre: server umount lustre-OST0000 complete [ 4903.910095] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4903.927824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4903.934851] Lustre: Skipped 1 previous similar message [ 4907.439086] Lustre: server umount lustre-MDT0000 complete [ 4909.027888] LustreError: 151529:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4909.046877] LustreError: 151529:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4911.189735] LustreError: 153020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774879289 with bad export cookie 11020038445821782924 [ 4911.193716] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4911.201449] LustreError: 153020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4911.505258] Lustre: server umount lustre-MDT0001 complete [ 4922.579578] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 4925.294857] Key type lgssc unregistered [ 4925.617400] LNet: 153755:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4925.627171] LNetError: 153755:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4926.698038] LNet: Removed LNI 192.168.204.115@tcp [ 4927.580423] Key type .llcrypt unregistered [ 4927.589159] Key type ._llcrypt unregistered [ 4953.024837] Key type ._llcrypt registered [ 4953.027843] Key type .llcrypt registered [ 4953.160241] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 4954.346743] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4954.485335] alg: No test for adler32 (adler32-zlib) [ 4955.644568] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 4955.969424] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 4957.642472] Key type lgssc registered [ 4958.984978] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4975.437752] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:02:31 (1774879351) [ 4993.863911] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 5008.712849] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 5049.338477] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5049.368245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5050.891849] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5050.945580] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5051.057880] Lustre: lustre-MDT0000: new disk, initializing [ 5051.252910] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5051.294734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5057.021454] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5071.211668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5071.431148] Lustre: 157992:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5071.531760] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5071.545654] Lustre: Skipped 1 previous similar message [ 5071.669294] Lustre: lustre-MDT0001: new disk, initializing [ 5071.807552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5071.848101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5071.862415] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5076.859083] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5086.357370] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5091.632755] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5099.435761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5099.791549] Lustre: lustre-OST0000: new disk, initializing [ 5099.809503] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5099.943670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5106.212861] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5108.313211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5108.337100] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5108.455133] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5113.770683] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5124.066976] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5124.085912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.110141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5124.119391] Lustre: Skipped 1 previous similar message [ 5129.197856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5129.207700] Lustre: Skipped 1 previous similar message [ 5130.121664] Lustre: server umount lustre-OST0000 complete [ 5138.402056] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5138.414789] LustreError: Skipped 1 previous similar message [ 5138.417023] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5138.434074] Lustre: Skipped 1 previous similar message [ 5138.447311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5139.428927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5141.438693] Lustre: server umount lustre-MDT0000 complete [ 5145.282890] LustreError: 158779:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774879523 with bad export cookie 15763659885921890327 [ 5145.285589] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5145.294676] LustreError: 158779:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5145.725783] Lustre: server umount lustre-MDT0001 complete [ 5160.388779] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 5164.720145] Key type lgssc unregistered [ 5165.195086] LNet: 161511:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5165.210353] LNetError: 161511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5165.249894] LNet: Removed LNI 192.168.204.115@tcp [ 5166.595073] Key type .llcrypt unregistered [ 5166.600949] Key type ._llcrypt unregistered [ 5196.534354] Key type ._llcrypt registered [ 5196.542454] Key type .llcrypt registered [ 5196.683881] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 5198.406749] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5198.489812] alg: No test for adler32 (adler32-zlib) [ 5199.628077] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 5199.982250] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 5201.768203] Key type lgssc registered [ 5203.423535] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5213.060395] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5213.111074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5214.684507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5219.831131] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5224.936253] LustreError: 162911:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5224.954481] LustreError: 162911:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5230.050920] LustreError: 162910:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5230.744049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5231.222740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5236.227112] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5242.390439] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5247.448616] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5255.119503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5255.472929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5259.601172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5261.786257] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 5267.988857] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5275.204743] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5275.229575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5275.254508] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5275.261985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.273901] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.464042] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.489216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.504313] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.518006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.546393] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5277.744369] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5277.789372] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5277.802573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5277.809733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.825794] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5278.986337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.002189] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5280.010433] Lustre: Skipped 1 previous similar message [ 5280.051917] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.067744] Lustre: Skipped 1 previous similar message [ 5280.073822] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5280.091460] Lustre: Skipped 1 previous similar message [ 5280.113741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.124315] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5280.127504] Lustre: Skipped 1 previous similar message [ 5281.086759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.017662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.233600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.297125] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5284.304987] Lustre: Skipped 3 previous similar messages [ 5284.318872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5284.325621] Lustre: Skipped 3 previous similar messages [ 5284.328460] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5284.331266] Lustre: Skipped 3 previous similar messages [ 5284.335617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.342191] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5284.350079] Lustre: Skipped 3 previous similar messages [ 5285.511745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.618551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.894926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.950271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.903358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.114699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.189730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.407847] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5293.417263] Lustre: Skipped 7 previous similar messages [ 5293.444546] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5293.455334] Lustre: Skipped 7 previous similar messages [ 5293.459582] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5293.463931] Lustre: Skipped 7 previous similar messages [ 5293.467712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.478668] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5293.487398] Lustre: Skipped 7 previous similar messages [ 5294.576511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.651941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.701606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.940808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.914228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.952946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.127600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.104402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.064788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.841903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.922585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.177878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.336234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.543617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.572900] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5309.590131] Lustre: Skipped 14 previous similar messages [ 5309.624689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5309.650232] Lustre: Skipped 14 previous similar messages [ 5309.661408] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5309.666588] Lustre: Skipped 14 previous similar messages [ 5309.673834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.704117] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5309.722310] Lustre: Skipped 14 previous similar messages [ 5310.579826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.744884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.674359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.870919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.051478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.089583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.941714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.095790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.106119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.229922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.227068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.261648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.438934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.450430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.643650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.814747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.895578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.003937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.255694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.407452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.643120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.576319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.650435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.635742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.604829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.749882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.834466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.800989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.761802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.775983] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5341.780583] Lustre: Skipped 29 previous similar messages [ 5341.800603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5341.809257] Lustre: Skipped 29 previous similar messages [ 5341.812857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5341.821262] Lustre: Skipped 29 previous similar messages [ 5341.824364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.837919] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5341.844185] Lustre: Skipped 29 previous similar messages [ 5342.848939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.780965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.689225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.680335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.624296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.740891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.750104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.846671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.900643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.945594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.010340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.233338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.315431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.131608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.032145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.940417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.054804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.129381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.093213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.209315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.175537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.437101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.537115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.628618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.549046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.662897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.682879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.615439] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5370.634151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.635269] LustreError: 162930:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5371.340707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.381928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.334917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.452152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.446407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.671799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.636967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.876796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.005486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.220898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.586176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.206643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.118861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.259961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.243394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.298150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.567119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.579613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.687291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.953803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.305351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.529498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.857735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.528299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.950950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.966303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.897612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.031140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.202791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.437995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.436559] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5406.447691] Lustre: Skipped 58 previous similar messages [ 5406.478929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5406.504168] Lustre: Skipped 58 previous similar messages [ 5406.510356] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5406.519987] Lustre: Skipped 58 previous similar messages [ 5406.541979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.562993] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5406.568151] Lustre: Skipped 58 previous similar messages [ 5407.834842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.146463] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5409.168259] LustreError: 162930:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5409.182497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.739516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.879258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.938278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.870502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.894817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.955574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.104593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.339075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.337315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.594617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.551246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.634554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.401397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.040656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.200895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.274931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.241884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.248844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.174210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.295979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.244665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.356346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.614284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.604251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.720564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.824171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.097584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.275284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.438317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.644955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.805425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.858160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.797855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.736423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.665031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.525837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.498239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.614920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.676228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.559538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.343291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.296853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.360833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.245200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.319141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.566615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.632521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.874299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.062698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.249937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.417868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.619652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.006079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.090942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.184247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.530857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.843535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.121991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.161308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.327022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.587969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.570884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.555957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.557495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.515895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.446821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.373694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.332140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.467165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.501208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.408585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.282278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.350483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.120190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.978700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.836420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.665391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.457598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.596527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.776538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.008406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.317557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.506048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.516412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.526089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.444445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.431228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.497873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.535075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.544373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.555463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.687625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.541784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.510665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.579709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.620902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.790799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.891789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.967105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.018972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.999632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.300865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.406299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.643445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.798304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.011467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.103381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.362299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.529502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.462951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.628482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.693233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.884928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.465521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.734736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.864465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.779805] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5534.784644] Lustre: Skipped 118 previous similar messages [ 5534.794930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5534.805829] Lustre: Skipped 118 previous similar messages [ 5534.810985] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5534.819324] Lustre: Skipped 118 previous similar messages [ 5534.823756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.834647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5534.841296] Lustre: Skipped 118 previous similar messages [ 5536.033306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.296930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.383980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.310857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.356963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.600504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.806489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.119498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.313817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.280554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.470893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.551863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.740643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.508927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.385115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.624333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.079667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.268430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.692509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.009614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.180084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.232533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.211114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.282418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.379506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.580550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.698363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.601374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.965557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.975475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.106165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.236624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.603873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.015357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.573738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.789405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.208358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.737269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.474120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.755177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.111627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.750447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.415721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.854840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.102183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.445818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.349770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.436575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.635237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.005249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.128143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.527932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.873840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.242302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.358734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.736624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.928682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.321888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.636920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.670070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.426752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.911760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.222569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.144415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.085430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.150632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.116275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.031412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.066948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.044878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.007874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.949668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.080557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.120102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.951600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.815077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.745237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.654149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.732371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.661847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.666921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.497603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.613255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.642582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.629706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.633445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.528929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.545583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.694030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.755418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.902177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.828910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.911930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.877130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.810502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.798604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.785549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.789608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.772776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.696123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.789706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.746217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.762381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.761397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.937186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.063337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.180883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.186710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.410822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.663798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.760600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.706987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.694984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.694074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.592453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.602757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.669482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.557831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.421149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.573465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.496571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.685581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.677144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.708483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.670860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.725826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.620540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.632486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.554622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.535499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.721654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.779273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.781244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.591339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.541043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.693690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.807813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.732352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.985585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.941694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.267047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.446087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.446178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.703627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.843520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.936679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.855110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.731901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.448732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.368649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.362286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.283851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.290365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.101552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.030451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.042544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.016671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.100558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.098475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.132440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.113768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.027764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.959701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.040792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.949891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.984202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.912494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.938642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.075352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.134448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.297991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.299798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.340635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.311852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.302561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.379862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.383337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.462180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.526676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.501379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.626695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.815757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.957641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.944875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.037717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.920747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.947362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.172932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.265310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.072996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.261770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.130426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.285173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.241701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.408541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.303352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.377897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.377080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.496277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.744717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.138188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.491725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.671140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.972504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.823536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.719365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.682397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.700721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.874564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.819176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.780617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.079965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.483774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.403927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.817703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.729524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.902450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.200968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.515564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.684662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.685637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.930669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.084884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.028162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.044376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.083770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.960688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.824064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.952649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.923286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.788887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.975926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.122961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.061994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.254523] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5791.261413] Lustre: Skipped 234 previous similar messages [ 5791.285268] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5791.299356] Lustre: Skipped 234 previous similar messages [ 5791.309059] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5791.315798] Lustre: Skipped 234 previous similar messages [ 5791.322535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.331931] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5791.344187] Lustre: Skipped 234 previous similar messages [ 5792.215902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.372995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.234585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.202123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.184612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.380301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.372705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.257938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.115696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.329221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.184832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.127096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.042752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.144291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.075595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.995655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.814197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.719861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.528143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.478052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.489865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.432218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.306165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.223552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.227465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.209327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.269422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.238479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.040775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.013587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.893297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.895283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.746065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.641716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.458274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.427341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.299880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.161330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.967442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.835494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.810991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.660850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.586713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.774152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.735724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.802863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.758353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.890793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.061077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.029889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.068580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.000391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.024667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.192119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.231135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.317197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.407108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.431485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.786452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.392912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.608805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.953154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.291723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.596117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.894029] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5856.922417] LustreError: 162930:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5856.951822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.732602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.194832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.469774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.437146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.450459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.372632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.282518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.260113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.249334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.150222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.092538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.110748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.115325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.235121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.133698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.169741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.119316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.044442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.144086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.049768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.176045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.283083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.264417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.325549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.315426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.499499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.450783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.619642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.612972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.983186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.279304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.498150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.486026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.452621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.289218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.296704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.375366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.557746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.711579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.892937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.231744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.436950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.834126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.943053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.953861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.089387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.184203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.608497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.913799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.334664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.698113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.923212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.787200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.707895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.758422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.114318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.209659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.299799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.486881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.734705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.940984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.797972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.870264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.839105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.982316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.994098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.155483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.125583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.239017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.229093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.831228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.972561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.986710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.169696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.265342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.260323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.334760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.501840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.596517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.554865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.581821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.566491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.543678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.482897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.437243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.464823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.479775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.273685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.257375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.110611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.036355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.224956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.396871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.643814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.629201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.829792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.039496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.549521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.521247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.281461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.596560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.877779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.016522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.073082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.110478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.453867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.608665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.888059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.353444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.943518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.390566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.430866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.452600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.348823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.548586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.555798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.652164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.586294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.639481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.715773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.605741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.438400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.369867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.348757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.597321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.671316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.644356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.658222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.859838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.898370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.025290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.787684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.709866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.667685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.542072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.610687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.577573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.589667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.782872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.869432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.147314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.231285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.141100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.139470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.001703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.907616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.027570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.059585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.033111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.989624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.900878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.617728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.429570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.303358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.257418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.162879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.108191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.935507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.794442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.851918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.675752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.906683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.115264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.120930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.184455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.019423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.116197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.191315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.220067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.054453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.989119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.837795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.731060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.610756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.505228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.450102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.641462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.559603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.542874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.450691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.422271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.545612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.733984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.821201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.671268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.685457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.456366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.416647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.263430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.238178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.210357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.199447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.071465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.974590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.913401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.747760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.529505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.450471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.244970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.095378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.989697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.909067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.765403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.758992] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6071.778192] LustreError: 162930:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6071.792122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.823657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.850908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.083358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.007218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.975235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.895566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.864761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.717671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.684311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.505206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.554729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.462450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.384651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.415106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.514070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.688725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.870255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.852680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.723456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.660921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.803482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.698090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.789905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.641237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.602575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.772990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.729153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.743689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.768342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.579159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.514319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.596788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.478261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.597048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.438231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.548848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.630174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.675991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.765847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.747551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.820222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.746268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.749230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.483235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.456093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.664656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.751317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.809642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.765507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.888340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.136465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.278315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.488952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.465148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.427807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.611674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.783983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.950420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.840221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.871128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.888521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.814934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.832524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.923274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.417558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.457389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.713248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.884275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.049160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.850362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.978627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.945292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.229331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.477896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.916062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.110392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.357172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.241722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.337820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.346779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.529813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.740436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.698438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.880849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.179429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.062368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.178084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.191408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.151094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.067226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.236847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.108247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.411564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.662896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.794717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.854734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.213608] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6174.232847] LustreError: 162930:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6174.255168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.940385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.526331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.999375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.301290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.575723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.815461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.356699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.714022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.267636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.840589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.881940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.755610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.714245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.905981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.995590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.034832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.819377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.626463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.094275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.411226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.315760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.303936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.471526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.417387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.424305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.419465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.727319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.789651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.819364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.182140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.477259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.704275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.794494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.496902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.014079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.373827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.425671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.449334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.305907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.385233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.456437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.232249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.078567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.009867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.999504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.890720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.910783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.909515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.838993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.895304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.733348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.672330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.577476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.559114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.384932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.266238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.138105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.056936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.909229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.805954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.714568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.621514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.457783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.500583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.392956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.341037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.308438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.288343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.246765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.524125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.538651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.332172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.335610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.239975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.078681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.046845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.221415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.245276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.281315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.212222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.213956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.134415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.046731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.023702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.239445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.298156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.353813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.354492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.365607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.737985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.860806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.793595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.730218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.821122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.751516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.921641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.024988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.035100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.264628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.232147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.163507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.143128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.798736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.536862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.518543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.366446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.280456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.924350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.159172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.343248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.411538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.526907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.332197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.104625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.027729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.025677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.272278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.008661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.046158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.961345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.884082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.661608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6303.664249] Lustre: Skipped 487 previous similar messages [ 6303.676806] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6303.682848] Lustre: Skipped 487 previous similar messages [ 6303.685432] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6303.687963] Lustre: Skipped 487 previous similar messages [ 6303.690902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.696976] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6303.699963] Lustre: Skipped 487 previous similar messages [ 6304.863845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.768616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.743679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.778673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.817872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.949491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.817063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.807393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.656554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.619193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.532739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.551391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.337574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.261921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.232181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.293160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.063284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.971930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.927106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.798560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.621538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.489332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.276831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.168488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.034541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.986411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.992722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.896445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.677927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.556270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.473779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.443028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.497313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.423679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.361069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.447383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.442740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.583452] LustreError: 164507:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6882.599151] LustreError: 162930:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7738.850778] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7738.876018] Lustre: Skipped 37 previous similar messages [ 7738.888968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7743.458176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7743.465090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7744.730197] Lustre: server umount lustre-OST0000 complete [ 7749.096622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7749.109293] Lustre: Skipped 2 previous similar messages [ 7754.211124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7754.220080] Lustre: Skipped 1 previous similar message [ 7755.387965] Lustre: server umount lustre-MDT0000 complete [ 7759.335114] LustreError: 162910:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7759.366445] LustreError: 162910:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 7760.118575] LustreError: 196435:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774882137 with bad export cookie 14209264137109029175 [ 7760.131444] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7760.662333] Lustre: server umount lustre-MDT0001 complete [ 7773.101475] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 7776.150464] Key type lgssc unregistered [ 7776.492679] LNet: 217876:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7776.504153] LNetError: 217876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7776.517044] LNet: Removed LNI 192.168.204.115@tcp [ 7777.273905] Key type .llcrypt unregistered [ 7777.285553] Key type ._llcrypt unregistered [ 7806.895128] Key type ._llcrypt registered [ 7806.899249] Key type .llcrypt registered [ 7807.050423] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 7808.424001] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7808.643723] alg: No test for adler32 (adler32-zlib) [ 7809.862614] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 7810.167107] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 7811.944195] Key type lgssc registered [ 7813.538460] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7823.711174] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7823.740035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7825.449084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7830.770592] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 7835.618292] LustreError: 219278:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7835.654703] LustreError: 219278:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 7840.746412] LustreError: 219279:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7841.759285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7842.140186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7847.839625] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 7855.690947] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7860.590675] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7868.585671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7869.069696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7871.159967] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 7877.909582] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 7887.528657] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7893.141583] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7896.041535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7896.054561] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7896.075850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7896.107192] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7896.108028] LustreError: 219298:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7899.147419] Lustre: lustre-OST0000: Not available for connect from 192.168.204.15@tcp (stopping) [ 7902.177752] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7902.180666] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7902.215624] Lustre: Skipped 1 previous similar message [ 7904.229183] Lustre: lustre-OST0000: Not available for connect from 192.168.204.15@tcp (stopping) [ 7904.243462] Lustre: Skipped 1 previous similar message [ 7905.030962] Lustre: server umount lustre-OST0000 complete [ 7916.112151] LustreError: 219298:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7940.877399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7941.287639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7942.332656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.359404] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.360318] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7942.388503] Lustre: Skipped 1 previous similar message [ 7947.655051] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 7953.762540] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7957.423288] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 7961.892827] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88f1d0e93800.ost_server_uuid 50 [ 7963.968277] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff88f1d0e93800.ost_server_uuid in FULL state after 0 sec [ 7969.260904] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7969.272867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7969.287321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7971.067424] Lustre: server umount lustre-OST0000 complete [ 7978.469864] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7978.477731] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7978.499310] Lustre: Skipped 1 previous similar message [ 7978.520740] Lustre: Skipped 1 previous similar message [ 7982.433474] Lustre: server umount lustre-MDT0000 complete [ 7983.589948] LustreError: 219274:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7983.612482] LustreError: 219274:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 7986.549299] LustreError: 219260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774882364 with bad export cookie 1204308670171157081 [ 7986.561171] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7986.565292] LustreError: 219260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7997.142729] Lustre: server umount lustre-MDT0001 complete [ 8010.822464] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 8014.544820] Key type lgssc unregistered [ 8015.184569] LNet: 223936:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8015.201975] LNetError: 223936:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8015.230790] LNet: Removed LNI 192.168.204.115@tcp [ 8016.625456] Key type .llcrypt unregistered [ 8016.629952] Key type ._llcrypt unregistered [ 8057.369621] Key type ._llcrypt registered [ 8057.374349] Key type .llcrypt registered [ 8057.518390] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 8058.836807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8059.021093] alg: No test for adler32 (adler32-zlib) [ 8060.179451] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 8060.387417] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 8062.057689] Key type lgssc registered [ 8063.458893] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8074.804298] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8074.818939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8076.564885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8082.670701] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8087.024807] LustreError: 225651:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8087.049049] LustreError: 225651:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 8092.132922] LustreError: 225650:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8093.422524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8094.112274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8099.772244] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8103.096551] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8111.130337] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8111.540803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8117.850951] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8121.793920] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 8127.260267] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8127.489891] Lustre: lustre-OST0001: new disk, initializing [ 8127.503215] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8127.636819] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8134.230205] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8135.287711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8135.296681] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8135.455338] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8146.877819] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8156.031389] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8181.224519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8181.231573] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8181.246510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8181.736333] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8181.748982] Lustre: Skipped 1 previous similar message [ 8181.756769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8181.765362] Lustre: Skipped 2 previous similar messages [ 8183.364778] Lustre: server umount lustre-MDT0000 complete [ 8186.857562] LustreError: 227881:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8186.869829] LustreError: 227881:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 8187.343142] LustreError: 228841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774882565 with bad export cookie 9012073970182636543 [ 8187.352915] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8187.362173] LustreError: 228841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8187.795633] Lustre: server umount lustre-MDT0001 complete [ 8202.843161] Lustre: server umount lustre-OST0000 complete [ 8217.137495] Lustre: server umount lustre-OST0001 complete [ 8242.730717] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 8258.057438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8258.900900] LustreError: 230991:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8259.112727] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8264.162533] LustreError: 230992:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8264.204476] LustreError: 230992:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 8264.437600] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8273.383200] LustreError: 230991:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.430676] LustreError: 230991:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 8276.712334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8277.032944] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8282.060210] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8285.644418] Lustre: 232100:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8294.248335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8294.687764] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8301.865909] LustreError: 232456:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8302.384952] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8303.981565] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 8313.254661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8322.313968] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8332.029462] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8336.987314] Lustre: 233948:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8354.788312] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8354.795572] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8354.805279] Lustre: Skipped 2 previous similar messages [ 8354.814597] Lustre: Skipped 3 previous similar messages [ 8359.911341] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8359.918727] Lustre: Skipped 3 previous similar messages [ 8360.379317] Lustre: server umount mylustre-MDT0000 complete [ 8365.029874] LustreError: 230992:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8365.053252] LustreError: 230992:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 7 previous similar messages [ 8365.403030] LustreError: 230966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774882743 with bad export cookie 9012073970182639350 [ 8365.404896] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8365.419036] LustreError: 230966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8365.989981] Lustre: server umount mylustre-MDT0001 complete [ 8381.611529] Lustre: server umount mylustre-OST0000 complete [ 8385.442783] Lustre: 224911:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774882747/real 1774882747] req@ffff8a73e3135f80 x1861099127478528/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1774882763 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8385.475059] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8385.495831] Lustre: Skipped 2 previous similar messages [ 8386.794865] Lustre: server umount mylustre-OST0001 complete [ 8413.998401] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 8427.744948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8428.542518] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8428.548442] Lustre: Skipped 1 previous similar message [ 8433.583818] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8433.635751] LustreError: 236246:0:(ldlm_lib.c:1178:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8433.655297] LustreError: 236246:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 8445.779630] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8453.138299] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8457.562658] Lustre: 237354:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8466.940686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8473.729579] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49793) [ 8474.166885] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8486.390933] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8488.467176] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8496.657389] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8505.299294] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8509.855652] Lustre: 239211:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8523.245190] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8523.257625] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8523.278988] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8523.413643] Lustre: server umount tfs-MDT0000 complete [ 8528.017943] LustreError: 236222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774882905 with bad export cookie 9012073970182641016 [ 8528.031459] LustreError: 236222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8528.043263] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8528.519387] Lustre: server umount tfs-MDT0001 complete [ 8543.868491] Lustre: server umount tfs-OST0000 complete [ 8548.836281] Lustre: 224912:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774882910/real 1774882910] req@ffff8a73e8189f80 x1861099127538816/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1774882926 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8548.873920] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8548.886968] Lustre: Skipped 3 previous similar messages [ 8549.391548] Lustre: server umount tfs-OST0001 complete [ 8572.931751] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 8584.852918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8585.405076] LustreError: 241443:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8585.433510] LustreError: 241443:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 13 previous similar messages [ 8585.562724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8585.569289] Lustre: Skipped 3 previous similar messages [ 8590.864529] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8602.630844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8608.639637] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8612.544687] Lustre: 242554:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8621.027312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8629.796334] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8630.663195] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49825) [ 8641.087875] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8642.605362] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8648.448697] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8657.324611] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8661.307904] Lustre: 244398:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8675.857390] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:04:11 (1774883051) [ 8692.718613] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8692.723803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8692.755682] Lustre: Skipped 2 previous similar messages [ 8692.760744] Lustre: Skipped 1 previous similar message [ 8698.596402] Lustre: server umount lustre-MDT0000 complete [ 8702.435129] LustreError: 241418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774883080 with bad export cookie 9012073970182642682 [ 8702.436936] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8702.442550] LustreError: 241418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8708.957851] Lustre: server umount lustre-MDT0001 complete [ 8713.189987] Lustre: server umount lustre-OST0000 complete [ 8717.690792] Lustre: server umount lustre-OST0001 complete [ 8724.978761] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 8735.454508] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 8780.076268] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8780.076268] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8780.076268] [ 8780.116357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8780.336161] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8780.369863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8780.472937] Lustre: lustre-MDT0000: new disk, initializing [ 8780.611919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8780.617263] Lustre: Skipped 3 previous similar messages [ 8780.630708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8786.332733] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8798.686829] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8798.686829] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8798.686829] [ 8798.779086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8799.082081] Lustre: 247549:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8799.229677] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8799.257517] Lustre: Skipped 1 previous similar message [ 8799.478940] Lustre: lustre-MDT0001: new disk, initializing [ 8799.673287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8799.688893] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8805.936910] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8820.039747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8820.293355] Lustre: lustre-OST0000: new disk, initializing [ 8820.302367] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8822.037192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8822.058644] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8822.210455] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8828.179897] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8840.341759] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8840.668172] Lustre: lustre-OST0001: new disk, initializing [ 8840.679249] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8842.230880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8842.238668] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8842.404884] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8848.501075] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 8862.984110] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:07:18 (1774883238) [ 8882.670916] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:07:38 (1774883258) [ 8903.659141] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8903.665578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8903.694174] Lustre: Skipped 4 previous similar messages [ 8903.714659] Lustre: Skipped 11 previous similar messages [ 8913.376767] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8913.625688] Lustre: server umount lustre-OST0000 complete [ 8913.895945] LustreError: 249087:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8913.934796] LustreError: 249087:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 15 previous similar messages [ 8924.685675] Lustre: server umount lustre-MDT0000 complete [ 8929.205170] LustreError: 247543:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774883307 with bad export cookie 9012073970182644236 [ 8929.219475] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8929.222790] LustreError: 247543:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8935.629455] Lustre: server umount lustre-MDT0001 complete [ 8948.346733] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 8962.000939] Lustre: server umount lustre-OST0001 complete [ 8970.452206] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 8990.066904] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 9037.723843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9038.094170] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9038.135347] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9038.225355] Lustre: lustre-MDT0000: new disk, initializing [ 9038.298233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9038.303554] Lustre: Skipped 3 previous similar messages [ 9038.323205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9044.031323] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9059.078300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9059.289971] Lustre: 254451:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9059.425385] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9059.449216] Lustre: Skipped 1 previous similar message [ 9059.604647] Lustre: lustre-MDT0001: new disk, initializing [ 9059.876991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9059.882304] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9066.024653] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9075.384489] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9079.784977] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9088.672935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9089.104431] Lustre: lustre-OST0000: new disk, initializing [ 9089.113307] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9090.951051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9090.963449] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9091.156360] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9096.525500] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9104.892868] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9116.647274] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9116.650789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9116.656114] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9116.656123] Lustre: Skipped 10 previous similar messages [ 9116.687594] Lustre: Skipped 5 previous similar messages [ 9118.213573] Lustre: server umount lustre-OST0000 complete [ 9126.380909] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9126.392840] LustreError: Skipped 1 previous similar message [ 9137.122185] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9137.616191] Lustre: server umount lustre-MDT0000 complete [ 9142.245133] LustreError: 254441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774883520 with bad export cookie 9012073970182645706 [ 9142.247655] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9142.261076] LustreError: 254441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9142.841566] Lustre: server umount lustre-MDT0001 complete [ 9156.415969] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 9160.276945] Key type lgssc unregistered [ 9160.843687] LNet: 257828:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9160.857104] LNetError: 257828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9160.876936] LNet: Removed LNI 192.168.204.115@tcp [ 9162.300181] Key type .llcrypt unregistered [ 9162.303339] Key type ._llcrypt unregistered [ 9192.520639] Key type ._llcrypt registered [ 9192.528919] Key type .llcrypt registered [ 9192.671210] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 9194.035860] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9194.117897] alg: No test for adler32 (adler32-zlib) [ 9195.308593] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 9195.574224] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 9197.264475] Key type lgssc registered [ 9198.954874] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9211.794808] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9211.859642] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9214.003212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9220.519472] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9224.193645] LustreError: 259278:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9224.251058] LustreError: 259278:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 9229.282317] LustreError: 259279:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9234.384485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9234.405638] LustreError: 259278:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9235.223112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9242.137760] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9249.768276] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9255.064355] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9263.908400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9264.430190] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9271.835916] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9279.722176] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9281.071593] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9284.972725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9284.983122] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9290.224238] LustreError: 260874:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9290.257748] LustreError: 260874:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 9290.661451] Lustre: server umount lustre-OST0000 complete [ 9295.329610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9295.360462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9295.372758] Lustre: Skipped 2 previous similar messages [ 9300.453500] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9300.465693] Lustre: Skipped 1 previous similar message [ 9301.742490] Lustre: server umount lustre-MDT0000 complete [ 9305.588880] LustreError: 259273:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9305.608033] LustreError: 259273:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 9306.441055] LustreError: 259260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774883684 with bad export cookie 7675292581836900738 [ 9306.445091] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9306.449350] LustreError: 259260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9307.080624] Lustre: server umount lustre-MDT0001 complete [ 9319.948094] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 9322.859409] Key type lgssc unregistered [ 9323.196200] LNet: 262512:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9323.203678] LNetError: 262512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9323.226889] LNet: Removed LNI 192.168.204.115@tcp [ 9324.116434] Key type .llcrypt unregistered [ 9324.119214] Key type ._llcrypt unregistered [ 9341.692437] Key type ._llcrypt registered [ 9341.695303] Key type .llcrypt registered [ 9341.789301] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 9342.623202] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9342.686505] alg: No test for adler32 (adler32-zlib) [ 9343.761505] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 9343.983961] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 9345.648267] Key type lgssc registered [ 9346.816545] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9356.517429] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9356.533202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9372.064688] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9377.184218] LustreError: 263694:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.115@tcp: failed processing log, type 4: rc = -110 [ 9404.000190] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9411.401543] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9418.330937] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9420.765911] Lustre: server umount lustre-OST0000 complete [ 9440.638832] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [ 9444.032620] Key type lgssc unregistered [ 9444.396698] LNet: 265196:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9444.405062] LNetError: 265196:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9444.421190] LNet: Removed LNI 192.168.204.115@tcp [ 9445.311882] Key type .llcrypt unregistered [ 9445.315727] Key type ._llcrypt unregistered [ 9490.768639] Key type ._llcrypt registered [ 9490.771549] Key type .llcrypt registered [ 9490.879546] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 9492.593908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9492.978253] alg: No test for adler32 (adler32-zlib) [ 9494.235678] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [ 9494.519354] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [ 9496.224202] Key type lgssc registered [ 9497.638681] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9511.225920] loop0: detected capacity change from 0 to 204800000 [ 9511.464389] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9511.491147] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9512.032387] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9513.253372] loop0: detected capacity change from 0 to 204800000 [ 9513.477326] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9513.488843] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9513.893744] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9515.052627] loop0: detected capacity change from 0 to 204800000 [ 9515.207888] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9515.253458] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9515.392842] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9516.809760] loop0: detected capacity change from 0 to 204800000 [ 9517.034930] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9517.054684] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9517.207645] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9518.559351] loop0: detected capacity change from 0 to 204800000 [ 9518.587289] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9523.608551] loop0: detected capacity change from 0 to 204800000 [ 9523.645815] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9527.947399] loop0: detected capacity change from 0 to 204800000 [ 9527.974325] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9532.380415] loop0: detected capacity change from 0 to 204800000 [ 9532.416264] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9536.734166] loop0: detected capacity change from 0 to 204800000 [ 9536.845494] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9536.935589] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9537.002849] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9537.009798] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9537.017657] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9537.027539] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9537.037255] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9537.049480] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9537.065478] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9537.076873] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9537.086587] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9537.101385] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9537.113200] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9537.128037] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9537.144918] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9537.167220] Lustre: 268592:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9543.548924] Lustre: server umount MGS complete [ 9544.526238] loop0: detected capacity change from 0 to 204800000 [ 9544.628304] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9545.274885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9545.342360] Lustre: lustre-MDT0000: Aborting client recovery [ 9545.359156] LustreError: 268969:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 9546.175884] loop1: detected capacity change from 0 to 204800000 [ 9546.409570] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9546.512622] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9546.524914] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9546.538116] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9546.546883] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9546.557634] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9546.571930] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9546.583724] Lustre: 269026:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9546.608559] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9546.793440] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9546.836607] Lustre: lustre-MDT0001: Aborting client recovery [ 9547.760616] loop2: detected capacity change from 0 to 204800000 [ 9547.867729] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9547.942118] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9547.959321] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9547.983584] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9548.005643] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9548.024554] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9548.037825] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9548.044909] Lustre: 269106:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9548.173191] Lustre: Found index 0 for lustre-OST0000, updating log [ 9548.371453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9548.407134] Lustre: lustre-OST0000: Aborting recovery [ 9549.387819] loop3: detected capacity change from 0 to 204800000 [ 9549.513846] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9549.638351] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9549.659592] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9549.685152] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9549.703185] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9549.719456] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9549.731728] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9549.755232] Lustre: 269190:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9549.793552] Lustre: Found index 1 for lustre-OST0001, updating log [ 9549.949378] Lustre: lustre-OST0001: Aborting recovery [ 9555.484660] LustreError: 268970:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9556.469539] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9556.473833] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9556.491661] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9556.499581] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9556.500958] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9556.585557] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9559.574358] LustreError: 268937:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1063us: rc = -2 [ 9560.108966] LustreError: 268934:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 12103us: rc = -2 [ 9560.131899] LustreError: 268934:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 9576.930101] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9576.935788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9576.954770] Lustre: Skipped 3 previous similar messages [ 9576.971279] Lustre: Skipped 3 previous similar messages [ 9580.031231] Lustre: server umount lustre-MDT0000 complete [ 9581.359757] LustreError: 268920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774883959 with bad export cookie 13538202110672661686 [ 9581.372777] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9582.049812] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9582.051481] LustreError: 268934:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9582.055928] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9582.065720] Lustre: Skipped 1 previous similar message [ 9582.126261] LustreError: 268934:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 9586.081762] LustreError: 268935:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9586.082397] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9586.147260] Lustre: Skipped 2 previous similar messages [ 9587.890037] Lustre: server umount lustre-MDT0001 complete [ 9595.618595] Lustre: server umount lustre-OST0000 complete [ 9611.233784] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9611.510888] Lustre: server umount lustre-OST0001 complete [ 9621.814334] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:19:57 (1774883997) [ 9657.679103] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [ 9668.289289] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [ 9710.051087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9710.254873] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9710.287257] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9710.382834] Lustre: lustre-MDT0000: new disk, initializing [ 9710.492053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9710.498779] Lustre: Skipped 1 previous similar message [ 9710.513683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9714.836632] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9727.006352] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9727.202850] Lustre: 273250:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9727.308549] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9727.312745] Lustre: Skipped 1 previous similar message [ 9727.416320] Lustre: lustre-MDT0001: new disk, initializing [ 9727.492235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9727.518081] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9727.533492] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9732.994757] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9742.752953] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9747.074683] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9755.675448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9756.036307] Lustre: lustre-OST0000: new disk, initializing [ 9756.042791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9756.203444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9763.446169] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9763.468720] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9763.692430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9765.415172] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9776.331110] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9784.046944] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9798.559240] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9801.020488] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9805.571633] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9855.971680] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9855.974934] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9855.983280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9856.040029] Lustre: Skipped 1 previous similar message [ 9859.678576] Lustre: server umount lustre-OST0000 complete [ 9865.698650] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9865.711515] LustreError: Skipped 1 previous similar message [ 9865.715883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9865.744408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9865.754392] Lustre: Skipped 1 previous similar message [ 9870.521221] Lustre: server umount lustre-MDT0000 complete [ 9871.330628] LustreError: 273257:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9871.362116] LustreError: 273257:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 9874.555794] LustreError: 276336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774884252 with bad export cookie 13538202110672664416 [ 9874.567815] LustreError: 276336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9874.577189] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9874.919245] Lustre: server umount lustre-MDT0001 complete [ 9884.696639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9889.474962] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9895.213630] Lustre: server umount MGS complete [ 9908.226275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9908.825617] LustreError: 278678:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9908.858816] LustreError: 278691:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9908.964903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9913.677853] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9914.344811] LustreError: 278679:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9923.069545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9928.369689] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9934.228334] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9938.828075] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9945.103550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9945.392414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9945.398738] Lustre: Skipped 1 previous similar message [ 9951.325505] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [ 9956.495274] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9974.251130] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9974.252811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9974.258409] LustreError: Skipped 1 previous similar message [ 9974.259428] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9974.259436] Lustre: Skipped 2 previous similar messages [ 9974.277984] Lustre: Skipped 2 previous similar messages [ 9979.765528] Lustre: server umount lustre-OST0000 complete [ 9989.616608] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9989.643191] Lustre: Skipped 1 previous similar message [ 9991.735078] Lustre: server umount lustre-MDT0000 complete [ 9994.742864] LustreError: 278673:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9994.778251] LustreError: 278673:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 9996.142841] LustreError: 278660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774884373 with bad export cookie 13538202110672666257 [ 9996.152204] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9996.162751] LustreError: 278660:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9996.611501] Lustre: server umount lustre-MDT0001 complete [10010.157469] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [10013.157432] Key type lgssc unregistered [10013.528207] LNet: 282273:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10013.534699] LNetError: 282273:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10013.545912] LNet: Removed LNI 192.168.204.115@tcp [10014.323616] Key type .llcrypt unregistered [10014.329076] Key type ._llcrypt unregistered [10053.658797] Key type ._llcrypt registered [10053.663426] Key type .llcrypt registered [10053.767267] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [10067.913307] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [10068.805802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10068.893069] alg: No test for adler32 (adler32-zlib) [10069.971350] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [10070.218020] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [10071.928341] Key type lgssc registered [10073.051370] Lustre: Echo OBD driver; http://www.lustre.org/ [10109.213051] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10109.245118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10110.458058] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10110.529801] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10110.635337] Lustre: lustre-MDT0000: new disk, initializing [10110.742802] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10110.765752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10114.717188] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10124.868291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10124.958425] Lustre: 286198:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10125.009458] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10125.013600] Lustre: Skipped 1 previous similar message [10125.073242] Lustre: lustre-MDT0001: new disk, initializing [10125.154845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10125.195704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10125.207524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10128.955151] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10136.484224] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10140.218440] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10147.554302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10147.947878] Lustre: lustre-OST0000: new disk, initializing [10147.953179] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10148.078606] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10154.816698] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10156.634078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10156.659873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10156.815714] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10162.887620] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10170.706211] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10181.870957] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10192.184346] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10200.379845] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10244.067361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10244.068254] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10244.075827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10244.077329] Lustre: Skipped 1 previous similar message [10249.187993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10249.200030] Lustre: Skipped 2 previous similar messages [10254.314792] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10254.327026] Lustre: Skipped 1 previous similar message [10256.352232] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10256.475501] Lustre: server umount lustre-OST0000 complete [10263.521950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10263.529538] LustreError: Skipped 1 previous similar message [10263.533840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10263.544851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10264.565935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10269.666656] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10269.673753] Lustre: Skipped 2 previous similar messages [10274.272322] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10274.610289] Lustre: server umount lustre-MDT0000 complete [10278.638460] LustreError: 286192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774884656 with bad export cookie 9622646856493501531 [10278.642777] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10278.650347] LustreError: 286192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10279.038951] Lustre: server umount lustre-MDT0001 complete [10287.238538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10291.560339] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10296.254694] Lustre: server umount MGS complete [10307.581287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10308.072293] LustreError: 291776:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10308.090608] LustreError: 291776:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [10308.104352] LustreError: 291789:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10308.158964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10312.720771] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10313.188781] LustreError: 291777:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10317.282374] LustreError: 291776:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10320.564612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10320.858147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10324.995432] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10330.455098] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10334.598719] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10342.201113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10342.578358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10348.931963] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10355.606932] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10388.972437] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10388.975230] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10388.978276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10388.988621] Lustre: Skipped 1 previous similar message [10393.261868] Lustre: server umount lustre-OST0000 complete [10397.665875] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10397.675433] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10403.597544] Lustre: server umount lustre-MDT0000 complete [10404.322568] LustreError: 292785:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10404.344713] LustreError: 292785:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [10408.036324] LustreError: 291758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774884785 with bad export cookie 9622646856493503365 [10408.046682] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10408.058123] LustreError: 291758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10408.453092] Lustre: server umount lustre-MDT0001 complete [10419.456310] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [10422.264863] Key type lgssc unregistered [10422.597054] LNet: 295367:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10422.611835] LNetError: 295367:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10422.633809] LNet: Removed LNI 192.168.204.115@tcp [10423.469939] Key type .llcrypt unregistered [10423.472261] Key type ._llcrypt unregistered [10443.696933] Key type ._llcrypt registered [10443.699106] Key type .llcrypt registered [10443.818794] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [10457.814095] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [10458.741584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10458.940703] alg: No test for adler32 (adler32-zlib) [10460.119359] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [10460.458026] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [10462.216816] Key type lgssc registered [10463.321898] Lustre: Echo OBD driver; http://www.lustre.org/ [10494.327313] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [10512.694306] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10512.722892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10513.987831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10514.028682] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10514.096241] Lustre: lustre-MDT0000: new disk, initializing [10514.204588] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10514.215653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10518.273842] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10528.887210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10529.189435] Lustre: lustre-OST0000: new disk, initializing [10529.199734] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10529.216395] Lustre: Skipped 1 previous similar message [10529.302023] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10535.417845] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [10538.049575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10538.057704] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10538.089679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11492.685073] Lustre: server umount lustre-MDT0000 complete [11497.115740] LustreError: 298574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774885874 with bad export cookie 14544092989276441133 [11497.139509] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11506.830068] Lustre: server umount lustre-OST0000 complete [11530.447106] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [11538.566712] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [11579.605187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11580.018457] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11580.075821] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11580.201372] Lustre: lustre-MDT0000: new disk, initializing [11580.332483] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11580.363466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11586.294814] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11599.813754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11600.015420] Lustre: 303669:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11600.107164] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11600.113595] Lustre: Skipped 1 previous similar message [11600.276580] Lustre: lustre-MDT0001: new disk, initializing [11600.506387] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11600.575873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11600.595140] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11605.823566] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11614.834260] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11619.539965] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11627.504967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11627.915363] Lustre: lustre-OST0000: new disk, initializing [11627.927110] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11628.094543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11634.224962] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11637.368421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11637.381372] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11637.526809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11642.327700] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11647.084811] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11647.323996] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11651.452602] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11651.685215] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11658.217463] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11658.217712] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11658.227113] LustreError: Skipped 1 previous similar message [11658.241934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11658.246315] Lustre: Skipped 1 previous similar message [11659.804363] Lustre: server umount lustre-OST0000 complete [11663.335850] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11663.347808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11663.353846] Lustre: Skipped 2 previous similar messages [11663.369087] Lustre: Skipped 1 previous similar message [11668.450024] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11668.456623] Lustre: Skipped 1 previous similar message [11669.646514] Lustre: server umount lustre-MDT0000 complete [11673.129313] LustreError: 303662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774886050 with bad export cookie 14544092989279381952 [11673.141131] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11673.143318] LustreError: 303662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11673.435054] Lustre: server umount lustre-MDT0001 complete [11677.395188] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11679.307542] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:54:15 (1774886055) [11685.749967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11686.121826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11689.845978] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11691.493847] LustreError: 307581:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11691.519243] LustreError: 307581:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [11695.590113] LustreError: 307582:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11697.918656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11698.230182] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11702.604675] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11708.303542] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11712.740431] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11722.032771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11722.714232] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11731.835904] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11738.644365] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11744.942599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11745.090186] Lustre: lustre-OST0001: new disk, initializing [11745.099922] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11745.170352] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11750.928659] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11752.610224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11752.618866] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11752.673871] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11752.685446] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11753.706284] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11753.724867] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11754.727279] LustreError: 310508:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11756.772917] LustreError: 310508:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11756.785295] LustreError: 310508:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11759.978877] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11760.196929] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11760.936417] LustreError: 310508:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11760.948984] LustreError: 310508:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11763.491628] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88f1f581c000.ost_server_uuid 50 [11764.652752] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff88f1f581c000.ost_server_uuid in FULL state after 0 sec [11768.126951] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88f1f581c000.ost_server_uuid 50 [11769.122879] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11769.136183] LustreError: 310501:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11769.734516] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff88f1f581c000.ost_server_uuid in FULL state after 0 sec [11772.345297] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11786.753494] Lustre: lustre-OST0001: Not available for connect from 192.168.204.15@tcp (stopping) [11787.745336] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11787.750685] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11787.756468] LustreError: Skipped 1 previous similar message [11787.789043] Lustre: Skipped 1 previous similar message [11791.206198] Lustre: server umount lustre-OST0001 complete [11791.845055] LustreError: 310511:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11793.896378] LustreError: 309176:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11793.919821] LustreError: 309176:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11797.477129] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11797.492686] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11797.511562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11797.534324] Lustre: Skipped 4 previous similar messages [11799.013821] LustreError: 309176:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11799.041231] LustreError: 309176:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11803.398531] Lustre: server umount lustre-OST0000 complete [11809.254864] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11809.282979] Lustre: Skipped 1 previous similar message [11809.286180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11809.292850] Lustre: Skipped 2 previous similar messages [11813.468792] Lustre: server umount lustre-MDT0000 complete [11814.369996] LustreError: 307578:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11814.381209] LustreError: 307578:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11817.508175] LustreError: 307563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774886195 with bad export cookie 14544092989279382834 [11817.515256] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11817.520945] LustreError: 307563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11817.985412] Lustre: server umount lustre-MDT0001 complete [11829.712174] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [11832.769623] Key type lgssc unregistered [11833.110733] LNet: 312523:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11833.123329] LNetError: 312523:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11833.149936] LNet: Removed LNI 192.168.204.115@tcp [11834.005361] Key type .llcrypt unregistered [11834.007822] Key type ._llcrypt unregistered [11859.187412] Key type ._llcrypt registered [11859.189792] Key type .llcrypt registered [11859.299732] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [11860.181896] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11860.315580] alg: No test for adler32 (adler32-zlib) [11861.424246] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [11861.618229] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [11863.272245] Key type lgssc registered [11864.647422] Lustre: Echo OBD driver; http://www.lustre.org/ [11872.994317] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11873.021444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11874.526916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11878.759738] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11884.580800] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11885.032776] LustreError: 313926:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11885.055267] LustreError: 313926:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [11890.148734] LustreError: 313927:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11892.503836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11892.915089] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11897.886339] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11907.997229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11908.361880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11913.801423] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11917.626976] LustreError: 315476:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11919.469263] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11919.661483] LustreError: 315476:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11919.695591] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11919.696332] LustreError: 315476:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11924.978813] LustreError: 315476:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11925.032827] LustreError: 315476:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11926.673988] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11927.024842] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11932.179315] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11935.237683] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [11941.764870] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [11949.986252] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11950.189918] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12060.132711] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12060.149188] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12060.168228] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12060.173655] Lustre: Skipped 1 previous similar message [12064.694471] Lustre: server umount lustre-OST0001 complete [12065.262047] LustreError: 315479:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12065.274827] LustreError: 315479:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [12071.424766] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12071.435620] LustreError: Skipped 1 previous similar message [12071.444638] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12071.459441] Lustre: Skipped 1 previous similar message [12071.474249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12073.446322] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12073.461890] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12073.480828] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12075.498597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12075.501552] Lustre: Skipped 1 previous similar message [12076.646992] Lustre: server umount lustre-OST0000 complete [12082.149361] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12082.162587] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12082.188806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12086.982781] Lustre: server umount lustre-MDT0000 complete [12090.849916] LustreError: 313927:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12090.868802] LustreError: 313927:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [12090.900033] LustreError: 313907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774886468 with bad export cookie 13327704772676371116 [12090.901668] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12090.923124] LustreError: 313907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12091.312359] Lustre: server umount lustre-MDT0001 complete [12103.328927] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [12106.356784] Key type lgssc unregistered [12106.697793] LNet: 318500:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12106.709040] LNetError: 318500:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12106.731590] LNet: Removed LNI 192.168.204.115@tcp [12107.703746] Key type .llcrypt unregistered [12107.708583] Key type ._llcrypt unregistered [12151.774051] Key type ._llcrypt registered [12151.783404] Key type .llcrypt registered [12151.898533] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [12153.059298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12153.080634] alg: No test for adler32 (adler32-zlib) [12154.163537] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [12154.353236] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [12156.016256] Key type lgssc registered [12157.373979] Lustre: Echo OBD driver; http://www.lustre.org/ [12165.367355] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12165.391407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12166.766341] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12170.048838] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12176.358931] LustreError: 320778:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12176.379365] LustreError: 320778:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [12178.254136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12178.659140] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12183.306297] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12186.420896] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12193.073407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12193.391110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12199.344362] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12203.622838] LustreError: 322233:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12207.880915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12208.103215] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12212.222956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12212.226698] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12214.075375] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12224.840249] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12235.272708] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12253.160782] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12253.164634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12253.177472] Lustre: Skipped 2 previous similar messages [12253.187102] Lustre: Skipped 1 previous similar message [12258.279883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12258.295865] Lustre: Skipped 4 previous similar messages [12258.785857] Lustre: server umount lustre-MDT0000 complete [12262.335918] LustreError: 320758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774886640 with bad export cookie 6307794078718294921 [12262.339273] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12262.346438] LustreError: 320758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12262.567342] Lustre: server umount lustre-MDT0001 complete [12276.987233] Lustre: server umount lustre-OST0000 complete [12279.264252] Lustre: 320039:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1774886641/real 1774886641] req@ffff8a73e7809c00 x1861103420244480/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1774886657 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12279.291405] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12279.314966] Lustre: Skipped 1 previous similar message [12281.010922] Lustre: server umount lustre-OST0001 complete [12297.313316] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [12307.366964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12307.801363] LustreError: 325968:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12307.917682] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12311.980997] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12313.072635] LustreError: 325969:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12318.185335] LustreError: 325968:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12321.155020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12321.678574] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12326.159859] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12329.193437] Lustre: 327076:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12336.658922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12344.448299] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12346.469262] LustreError: 327429:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12354.051516] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12354.201386] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12354.412606] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12354.418906] Lustre: Skipped 1 previous similar message [12355.893864] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12360.487719] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12368.311228] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12371.955948] Lustre: 328919:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12383.201186] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12383.205167] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12383.216880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12385.248936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12385.254254] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12385.272031] Lustre: Skipped 2 previous similar messages [12387.348931] Lustre: server umount lustre-MDT0000 complete [12390.376420] LustreError: 325964:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12390.396953] LustreError: 325964:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 6 previous similar messages [12391.106909] LustreError: 325949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774886768 with bad export cookie 6307794078718296622 [12391.110300] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12391.116712] LustreError: 325949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12391.542961] Lustre: server umount lustre-MDT0001 complete [12405.825085] Lustre: server umount lustre-OST0000 complete [12420.285406] Lustre: server umount lustre-OST0001 complete [12424.189483] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12425.858083] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12428.147414] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:06:43 (1774886803) [12449.479983] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [12457.950066] loop0: detected capacity change from 0 to 1099511627776 [12458.566153] SGI XFS with ACLs, security attributes, quota, no debug enabled [12458.571583] *** VALIDATE xfs *** [12458.590724] XFS (loop0): Mounting V5 Filesystem [12458.637550] XFS (loop0): Ending clean mount [12459.999927] loop1: detected capacity change from 0 to 18691697672192 [12573.021956] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [12585.480328] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [12625.326336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12625.648696] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12625.693136] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12625.786477] Lustre: lustre-MDT0000: new disk, initializing [12625.878339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12625.907945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12631.335828] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12644.300558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12644.439070] Lustre: 334006:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12644.492228] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12644.499140] Lustre: Skipped 1 previous similar message [12644.605911] Lustre: lustre-MDT0001: new disk, initializing [12644.764994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12644.780061] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12650.010715] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12657.913803] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12661.913419] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12669.744680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12669.998765] Lustre: lustre-OST0000: new disk, initializing [12670.003315] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12671.370961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12671.392862] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12671.512074] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12676.687406] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12684.719409] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12689.796986] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12690.049483] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12694.838669] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12695.030509] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12702.183461] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12702.188948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12702.197651] LustreError: Skipped 1 previous similar message [12702.203086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12702.219515] Lustre: Skipped 1 previous similar message [12702.253696] Lustre: Skipped 4 previous similar messages [12711.392144] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12711.565418] Lustre: server umount lustre-OST0000 complete [12716.518323] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12716.527805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12716.540322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12716.549095] Lustre: Skipped 2 previous similar messages [12721.961496] Lustre: server umount lustre-MDT0000 complete [12722.657680] LustreError: 334863:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12722.687095] LustreError: 334863:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [12726.528490] LustreError: 334786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774887104 with bad export cookie 6307794078718298176 [12726.534817] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12726.548021] LustreError: 334786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12726.987697] Lustre: server umount lustre-MDT0001 complete [12737.680649] XFS (loop0): Unmounting Filesystem [12743.097558] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:11:59 (1774887119) [12750.190287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12751.209088] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12751.223618] Lustre: Skipped 2 previous similar messages [12756.855920] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12767.429939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12773.161956] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12779.084735] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12782.964116] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12789.728735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12795.514917] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12800.941590] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12810.723854] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12810.724363] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12810.755575] Lustre: Skipped 1 previous similar message [12810.776475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12810.787709] Lustre: Skipped 3 previous similar messages [12815.861157] LustreError: 340076:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12815.882099] LustreError: 340076:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [12816.027790] Lustre: server umount lustre-OST0000 complete [12834.272123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12834.445601] Lustre: server umount lustre-MDT0000 complete [12838.268441] LustreError: 338142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774887216 with bad export cookie 6307794078718299051 [12838.277758] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12838.288155] LustreError: 338142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12838.810215] Lustre: server umount lustre-MDT0001 complete [12850.918326] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [12854.338846] Key type lgssc unregistered [12854.739828] LNet: 341440:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12854.756443] LNetError: 341440:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12854.780903] LNet: Removed LNI 192.168.204.115@tcp [12855.844305] Key type .llcrypt unregistered [12855.847568] Key type ._llcrypt unregistered [12886.672227] Key type ._llcrypt registered [12886.674220] Key type .llcrypt registered [12886.777556] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [12888.146633] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12888.169932] alg: No test for adler32 (adler32-zlib) [12889.277721] Lustre: Lustre: Build Version: 2.17.51_23_gefc2bdf [12889.524593] LNet: Added LNI 192.168.204.115@tcp [8/256/0/180] [12891.256208] Key type lgssc registered [12892.485989] Lustre: Echo OBD driver; http://www.lustre.org/ [12901.894427] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12901.932647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12903.528493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12908.342262] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12913.642227] LustreError: 342995:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12913.660105] LustreError: 342995:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [12917.163977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12917.636294] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12921.749581] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12927.679463] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12931.536894] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12938.133936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12938.513457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12945.345659] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12951.591376] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12963.809485] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12963.815779] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12963.829299] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12964.331729] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12964.368544] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12964.372723] Lustre: Skipped 1 previous similar message [12966.496556] Lustre: server umount lustre-MDT0001 complete [12969.442702] LustreError: 344358:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12969.464522] LustreError: 344358:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [12974.565104] LustreError: 342989:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12974.599487] LustreError: 342989:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [12978.052965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12978.249297] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12978.258451] Lustre: Found index 1 for lustre-MDT0001, updating log [12978.346060] Lustre: 342983:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12978.812429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12983.924161] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [12984.302530] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12986.893101] LustreError: 346216:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12986.906616] LustreError: 346216:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12986.925663] LustreError: 342987:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12987.001786] LustreError: 346224:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.204.115@tcp: cfg command failed: rc = -17 [12987.009341] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12987.009341] [13291.492583] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13291.493991] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13291.512989] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13291.522281] Lustre: Skipped 1 previous similar message [13291.536867] Lustre: Skipped 1 previous similar message [13295.552215] Lustre: server umount lustre-MDT0001 complete [13296.623995] LustreError: 342995:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13296.650493] LustreError: 342995:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [13301.733153] LustreError: 342991:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13301.752928] LustreError: 342991:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [13307.293666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13307.437921] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13307.448198] Lustre: Found index 1 for lustre-MDT0001, updating log [13307.540195] Lustre: 342984:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13307.951551] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13313.020904] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13313.119151] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [13313.524560] LustreError: 354270:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13313.533215] LustreError: 354270:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13313.543936] LustreError: 342987:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13313.550047] LustreError: 342987:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13313.644975] LustreError: 354273:0:(obd_config.c:760:class_setup()) Device 8 already setup (type osp) [13313.652739] LustreError: 354273:0:(obd_config.c:864:class_detach()) OBD device 8 still set up [13313.662812] LustreError: 354273:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.204.115@tcp: cfg command failed: rc = -17 [13313.679925] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.115@tcp [13313.679925] [13619.211611] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.15@tcp (stopping) [13619.226497] Lustre: Skipped 1 previous similar message [13620.197093] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13620.198991] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13620.225812] Lustre: Skipped 1 previous similar message [13624.293341] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.15@tcp (stopping) [13624.308464] Lustre: Skipped 2 previous similar messages [13625.624080] Lustre: server umount lustre-MDT0001 complete [13629.412032] LustreError: 342991:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.15@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13629.437932] LustreError: 342991:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [13637.808301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13638.035812] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13638.055294] Lustre: Found index 1 for lustre-MDT0001, updating log [13638.110979] Lustre: 342984:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13638.605241] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13643.763645] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13644.224586] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [13647.890647] LustreError: 362613:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13647.899405] LustreError: 362613:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13647.928594] LustreError: 342987:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13647.937730] LustreError: 342987:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13648.043501] LustreError: 362616:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13648.058526] LustreError: 362616:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.204.115@tcp: cfg command failed: rc = -114 [13648.069033] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13648.069033] [13971.429044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13971.429069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13971.438851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13971.454336] Lustre: Skipped 4 previous similar messages [13973.989812] Lustre: server umount lustre-MDT0000 complete [13976.547132] LustreError: 342990:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13976.588687] LustreError: 342990:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 7 previous similar messages [13978.124729] LustreError: 342974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774888355 with bad export cookie 8175439942873614315 [13978.127362] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13978.134153] LustreError: 342974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13981.673678] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13981.697747] Lustre: Skipped 2 previous similar messages [13984.708970] Lustre: server umount lustre-MDT0001 complete [13988.854900] Lustre: server umount lustre-OST0000 complete [13998.106213] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [14008.417605] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [14048.482993] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:33:44 (1774888424) [14055.186567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14055.495987] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14055.560579] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14055.712590] Lustre: lustre-MDT0000: new disk, initializing [14055.817919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14055.846788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14060.867841] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [14071.484403] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14071.627777] Lustre: 373009:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14071.717114] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14071.722174] Lustre: Skipped 1 previous similar message [14071.813261] Lustre: lustre-MDT0001: new disk, initializing [14071.905075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14071.942797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14071.954521] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14076.698451] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [14085.416977] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14090.138940] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14097.373593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14097.625729] Lustre: lustre-OST0000: new disk, initializing [14097.630437] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14097.714195] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14104.175128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14104.209834] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14104.362810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14104.777842] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_default_debug -1 all [14114.488821] Lustre: DEBUG MARKER: oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14125.027365] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14125.041444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14125.046713] Lustre: Skipped 1 previous similar message [14125.069049] Lustre: Skipped 4 previous similar messages [14130.136850] Lustre: server umount lustre-MDT0000 complete [14130.153725] LustreError: 373016:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14130.184299] LustreError: 373016:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [14134.136113] LustreError: 372998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1774888511 with bad export cookie 8175439942873616268 [14134.140690] LustreError: MGC192.168.204.115@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14134.147701] LustreError: 372998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14134.646556] Lustre: server umount lustre-MDT0001 complete [14149.691419] Lustre: server umount lustre-OST0000 complete [14180.156758] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing set_hostid [14191.260194] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing load_modules_local [14227.443800] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13950 sec ========= 12:36:43 (1774888603) [14229.564763] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:36:45 (1774888605) === [14233.353288] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:36:49 (1774888609) === [14276.376511] Lustre: DEBUG MARKER: oleg415-server.virtnet: executing unload_modules_local [14279.287079] Key type lgssc unregistered [14279.585455] LNet: 380956:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14279.595440] LNetError: 380956:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14279.644688] LNet: Removed LNI 192.168.204.115@tcp [14280.435594] Key type .llcrypt unregistered [14280.437658] Key type ._llcrypt unregistered