[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 524767280 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003218] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005020] kvm-guest: setup PV IPIs [ 0.007688] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008027] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009016] pid_max: default: 32768 minimum: 301 [ 0.010152] LSM: Security Framework initializing [ 0.011072] Yama: becoming mindful. [ 0.012052] SELinux: Initializing. [ 0.014082] *** VALIDATE selinux *** [ 0.022445] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026827] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027183] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028131] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030128] *** VALIDATE tmpfs *** [ 0.031491] *** VALIDATE proc *** [ 0.032243] *** VALIDATE cgroup *** [ 0.033015] *** VALIDATE cgroup2 *** [ 0.034313] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035189] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036015] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037041] Spectre V2 : User space: Vulnerable [ 0.038014] Speculative Store Bypass: Vulnerable [ 0.041229] debug: unmapping init [mem 0xffffffff8b659000-0xffffffff8b660fff] [ 0.043995] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044750] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045031] ... version: 2 [ 0.046016] ... bit width: 48 [ 0.047018] ... generic registers: 4 [ 0.048017] ... value mask: 0000ffffffffffff [ 0.049021] ... max period: 00007fffffffffff [ 0.050023] ... fixed-purpose events: 3 [ 0.051021] ... event mask: 000000070000000f [ 0.052340] rcu: Hierarchical SRCU implementation. [ 0.054532] smp: Bringing up secondary CPUs ... [ 0.055712] x86: Booting SMP configuration: [ 0.056044] .... node #0, CPUs: #1 #2 #3 [ 0.063083] smp: Brought up 1 node, 4 CPUs [ 0.065020] smpboot: Max logical packages: 1 [ 0.066019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.273185] node 0 deferred pages initialised in 205ms [ 0.276197] devtmpfs: initialized [ 0.277261] x86/mm: Memory block size: 128MB [ 0.279891] gcov: version magic: 0x41383552 [ 0.280633] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.284129] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.287419] pinctrl core: initialized pinctrl subsystem [ 0.289210] [ 0.289739] ************************************************************* [ 0.293019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.294013] ** ** [ 0.296016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.298015] ** ** [ 0.300037] ** This means that this kernel is built to expose internal ** [ 0.303017] ** IOMMU data structures, which may compromise security on ** [ 0.305015] ** your system. ** [ 0.307018] ** ** [ 0.309016] ** If you see this message and you are not debugging the ** [ 0.311016] ** kernel, report this immediately to your vendor! ** [ 0.313014] ** ** [ 0.315014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.317013] ************************************************************* [ 0.319797] NET: Registered protocol family 16 [ 0.321499] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.324068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.326074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.330018] cpuidle: using governor menu [ 0.332009] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.334458] PCI: Using configuration type 1 for base access [ 0.336205] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.346208] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.348024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.351206] cryptd: max_cpu_qlen set to 1000 [ 0.355215] ACPI: Added _OSI(Module Device) [ 0.356035] ACPI: Added _OSI(Processor Device) [ 0.358025] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.359016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.364191] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.370353] ACPI: Interpreter enabled [ 0.372082] ACPI: PM: (supports S0 S3 S4 S5) [ 0.373017] ACPI: Using IOAPIC for interrupt routing [ 0.374142] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.377478] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.388631] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.391061] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.394038] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.397100] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.401426] acpiphp: Slot [2] registered [ 0.402127] acpiphp: Slot [5] registered [ 0.403134] acpiphp: Slot [6] registered [ 0.405128] acpiphp: Slot [7] registered [ 0.406125] acpiphp: Slot [8] registered [ 0.408118] acpiphp: Slot [9] registered [ 0.409171] acpiphp: Slot [10] registered [ 0.410130] acpiphp: Slot [3] registered [ 0.412117] acpiphp: Slot [4] registered [ 0.413134] acpiphp: Slot [11] registered [ 0.414137] acpiphp: Slot [12] registered [ 0.416120] acpiphp: Slot [13] registered [ 0.417134] acpiphp: Slot [14] registered [ 0.418143] acpiphp: Slot [15] registered [ 0.420125] acpiphp: Slot [16] registered [ 0.421134] acpiphp: Slot [17] registered [ 0.422144] acpiphp: Slot [18] registered [ 0.424133] acpiphp: Slot [19] registered [ 0.425153] acpiphp: Slot [20] registered [ 0.427158] acpiphp: Slot [21] registered [ 0.428149] acpiphp: Slot [22] registered [ 0.429134] acpiphp: Slot [23] registered [ 0.431128] acpiphp: Slot [24] registered [ 0.432136] acpiphp: Slot [25] registered [ 0.434148] acpiphp: Slot [26] registered [ 0.435147] acpiphp: Slot [27] registered [ 0.436159] acpiphp: Slot [28] registered [ 0.438153] acpiphp: Slot [29] registered [ 0.439131] acpiphp: Slot [30] registered [ 0.441144] acpiphp: Slot [31] registered [ 0.442080] PCI host bridge to bus 0000:00 [ 0.443031] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.446038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.448028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.450042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.452028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.455033] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.456200] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.458997] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.462371] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.472027] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.478774] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.481025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.483032] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.485021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.487489] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.489878] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.493052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.495883] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.501019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.515022] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.521021] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.527292] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.539023] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.545023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.569034] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.579000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.593030] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.607032] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.631024] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.644492] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.651026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.658028] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.680022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.688657] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.696022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.702024] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.719028] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.728149] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.737023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.744022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.764025] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.775887] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.781021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.787019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.802022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.814270] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.817436] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.819422] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.823353] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.825253] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.830110] iommu: Default domain type: Passthrough [ 0.831391] SCSI subsystem initialized [ 0.832120] ACPI: bus type USB registered [ 0.833191] usbcore: registered new interface driver usbfs [ 0.834088] usbcore: registered new interface driver hub [ 0.836091] usbcore: registered new device driver usb [ 0.837142] pps_core: LinuxPPS API ver. 1 registered [ 0.838011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.840058] PTP clock support registered [ 0.842103] EDAC MC: Ver: 3.0.0 [ 0.844232] PCI: Using ACPI for IRQ routing [ 0.847158] NetLabel: Initializing [ 0.848012] NetLabel: domain hash size = 128 [ 0.850014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.852102] NetLabel: unlabeled traffic allowed by default [ 0.854241] vgaarb: loaded [ 0.856315] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.858023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.862422] clocksource: Switched to clocksource kvm-clock [ 0.961822] VFS: Disk quotas dquot_6.6.0 [ 0.963341] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.965746] *** VALIDATE ramfs *** [ 0.966938] *** VALIDATE hugetlbfs *** [ 0.968616] pnp: PnP ACPI init [ 0.971147] pnp: PnP ACPI: found 6 devices [ 0.989170] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.991576] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.993653] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.995659] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.997353] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.998849] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.000828] NET: Registered protocol family 2 [ 1.002754] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.006570] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.009497] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.014048] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.016996] TCP: Hash tables configured (established 65536 bind 65536) [ 1.019217] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.021505] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.023738] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.025787] NET: Registered protocol family 1 [ 1.027751] RPC: Registered named UNIX socket transport module. [ 1.029167] RPC: Registered udp transport module. [ 1.030358] RPC: Registered tcp transport module. [ 1.031586] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.033155] NET: Registered protocol family 44 [ 1.034337] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.035957] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.037430] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.039163] PCI: CLS 0 bytes, default 64 [ 1.040387] Unpacking initramfs... [ 2.476131] debug: unmapping init [mem 0xffff8bd53cc54000-0xffff8bd53ffbffff] [ 2.479977] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.482457] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.485428] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.000931] Initialise system trusted keyrings [ 3.002658] Key type blacklist registered [ 3.004272] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.014777] zbud: loaded [ 3.018061] *** VALIDATE nfs *** [ 3.019410] *** VALIDATE nfs4 *** [ 3.021087] pstore: using deflate compression [ 3.024537] Platform Keyring initialized [ 3.112106] NET: Registered protocol family 38 [ 3.113367] Key type asymmetric registered [ 3.114270] Asymmetric key parser 'x509' registered [ 3.115799] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.118269] io scheduler mq-deadline registered [ 3.119919] io scheduler kyber registered [ 3.121989] io scheduler bfq registered [ 3.124088] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.126151] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.128069] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.130052] ACPI: Power Button [PWRF] [ 3.133438] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.138328] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.150224] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.157945] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.172594] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.199502] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.231586] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.239519] Non-volatile memory driver v1.3 [ 3.242740] Linux agpgart interface v0.103 [ 3.280810] virtio_blk virtio1: [vda] 149896 512-byte logical blocks (76.7 MB/73.2 MiB) [ 3.284491] vda: detected capacity change from 0 to 76746752 [ 3.307353] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.312993] vdb: detected capacity change from 0 to 1073741824 [ 3.336754] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.339324] vdc: detected capacity change from 0 to 2621440000 [ 3.378285] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.382825] vdd: detected capacity change from 0 to 2621440000 [ 3.406123] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.410746] vde: detected capacity change from 0 to 4294967296 [ 3.430488] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.434073] vdf: detected capacity change from 0 to 4294967296 [ 3.442572] libphy: Fixed MDIO Bus: probed [ 3.450916] usbcore: registered new interface driver usbserial_generic [ 3.453215] usbserial: USB Serial support registered for generic [ 3.455692] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.460662] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.462223] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.464672] mousedev: PS/2 mouse device common for all mice [ 3.469895] rtc_cmos 00:05: RTC can wake from S4 [ 3.474123] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.482091] rtc_cmos 00:05: registered as rtc0 [ 3.483734] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.484504] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.491592] intel_pstate: CPU model not supported [ 3.491807] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.499301] hid: raw HID events driver (C) Jiri Kosina [ 3.501438] usbcore: registered new interface driver usbhid [ 3.503322] usbhid: USB HID core driver [ 3.505099] drop_monitor: Initializing network drop monitor service [ 3.506988] Initializing XFRM netlink socket [ 3.508922] NET: Registered protocol family 10 [ 3.512498] Segment Routing with IPv6 [ 3.513926] NET: Registered protocol family 17 [ 3.515805] mpls_gso: MPLS GSO support [ 3.521969] RAS: Correctable Errors collector initialized. [ 3.524264] AVX version of gcm_enc/dec engaged. [ 3.526044] AES CTR mode by8 optimization enabled [ 3.606942] sched_clock: Marking stable (3606911483, 0)->(4572832202, -965920719) [ 3.610127] registered taskstats version 1 [ 3.612220] Loading compiled-in X.509 certificates [ 3.613678] zswap: loaded using pool lzo/zbud [ 3.640516] Key type big_key registered [ 3.654504] Key type encrypted registered [ 3.656253] ima: No TPM chip found, activating TPM-bypass! [ 3.658258] ima: Allocated hash algorithm: sha1 [ 3.659927] ima: No architecture policies found [ 3.661416] evm: Initialising EVM extended attributes: [ 3.663325] evm: security.selinux [ 3.664579] evm: security.ima [ 3.665795] evm: security.capability [ 3.667215] evm: HMAC attrs: 0x1 [ 3.669458] rtc_cmos 00:05: setting system clock to 2026-09-03 08:02:41 UTC (1788422561) [ 3.675346] debug: unmapping init [mem 0xffffffff8c603000-0xffffffff8c7fffff] [ 3.678238] debug: unmapping init [mem 0xffffffff8b382000-0xffffffff8b658fff] [ 3.686107] Write protecting the kernel read-only data: 28672k [ 3.689737] debug: unmapping init [mem 0xffffffff89a03000-0xffffffff89bfffff] [ 3.692361] debug: unmapping init [mem 0xffffffff8a314000-0xffffffff8a3fffff] [ 3.725578] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.734208] systemd[1]: Detected virtualization kvm. [ 3.736263] systemd[1]: Detected architecture x86-64. [ 3.738062] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.765643] systemd[1]: No hostname configured. [ 3.767038] systemd[1]: Set hostname to . [ 3.768639] random: systemd: uninitialized urandom read (16 bytes read) [ 3.770992] systemd[1]: Initializing machine ID from random generator. [ 3.814693] random: ln: uninitialized urandom read (6 bytes read) [ 3.921349] random: systemd: uninitialized urandom read (16 bytes read) [ 3.923654] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.928128] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.934919] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. Starting Apply Kernel Variables... Starting Setup Virtual Console... Starting Journal Service... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK [ 4.508160] random: fast init done ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.586873] device-mapper: uevent: version 1.0.3 [ 4.588691] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.328885] virtio_net virtio0 ens2: renamed from eth0 [ 5.404532] scsi host0: ata_piix [ 5.420496] scsi host1: ata_piix [ 5.422380] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.424877] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.164543] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.172345] random: crng init done [ 10.173770] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.619242] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.852923] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.121159] SELinux: Disabled at runtime. [ 12.187425] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.196185] systemd[1]: Detected virtualization kvm. [ 12.197574] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.735235] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.738278] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.746094] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.750907] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.753531] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.765420] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.772179] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. Mounting Kernel Debug File System... Starting Apply Kernel Variables... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Reached target Paths. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ 12.952759] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.231313] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.548908] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.558039] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.725288] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.737679] EDAC sbridge: Ver: 1.1.2 [ 15.328811] Key type dns_resolver registered [ 15.651249] NFS: Registering the id_resolver key type [ 15.652939] Key type id_resolver registered [ 15.654484] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg201-server login: [ 42.862336] libcfs: loading out-of-tree module taints kernel. [ 42.884650] Key type ._llcrypt registered [ 42.886574] Key type .llcrypt registered [ 42.930769] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 55.615921] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 57.804805] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 57.830337] alg: No test for adler32 (adler32-zlib) [ 59.800624] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 60.933920] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 62.811529] Key type lgssc registered [ 64.338434] hrtimer: interrupt took 8426714 ns [ 64.770500] Lustre: Echo OBD driver; http://www.lustre.org/ [ 89.019736] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 132.706175] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 148.875301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 148.911457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 150.391311] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 150.447447] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 150.583428] Lustre: lustre-MDT0000: new disk, initializing [ 150.754407] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 150.807271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 156.648240] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 173.946026] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 174.138637] Lustre: 6480:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 174.264526] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 174.276864] Lustre: Skipped 1 previous similar message [ 174.477473] Lustre: lustre-MDT0001: new disk, initializing [ 174.569227] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 174.610896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 174.620129] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 179.603319] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.736269] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 195.407125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 195.794991] Lustre: lustre-OST0000: new disk, initializing [ 195.805305] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 195.813828] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 195.815963] Lustre: 8420:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 195.931029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 201.199725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 201.210407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 201.303244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 204.414496] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.730429] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.078122] Lustre: lustre-OST0001: new disk, initializing [ 225.091130] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.100575] Lustre: 9488:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.271943] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 230.663033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 230.684141] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 230.725098] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 232.241568] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.604124] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 254.143340] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 262.628724] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing check_logdir /tmp/testlogs/ [ 270.547466] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing yml_node [ 275.355585] Lustre: DEBUG MARKER: Client: 2.17.58.7 [ 278.407958] Lustre: DEBUG MARKER: MDS: 2.17.58.7 [ 282.743408] Lustre: DEBUG MARKER: OSS: 2.17.58.7 [ 285.428650] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 04:07:20 EDT 2026 [ 304.417791] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 306.374917] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 323.072560] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 323.079536] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 323.113045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.181899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.200736] Lustre: Skipped 6 previous similar messages [ 329.165547] Lustre: server umount lustre-MDT0000 complete [ 334.724818] LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788422892 with bad export cookie 14197179033764645360 [ 334.733204] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 334.735217] LustreError: 6473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 338.400742] LustreError: 9496:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 338.407452] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.412740] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 338.445616] LustreError: 9496:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 338.518732] Lustre: Skipped 4 previous similar messages [ 339.424144] LustreError: 6489:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 344.544082] LustreError: 12233:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 344.545213] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 344.588243] LustreError: 12233:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 344.614053] Lustre: Skipped 4 previous similar messages [ 349.154321] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 349.314612] Lustre: server umount lustre-MDT0001 complete [ 353.796559] Lustre: server umount lustre-OST0000 complete [ 357.560771] Lustre: server umount lustre-OST0001 complete [ 365.087359] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 378.082428] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 424.025877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.517391] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 424.569370] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 424.692901] Lustre: lustre-MDT0000: new disk, initializing [ 424.785361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 424.822383] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 431.911840] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 444.337667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 444.488772] Lustre: 15048:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 444.704877] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 444.715223] Lustre: Skipped 1 previous similar message [ 444.855199] Lustre: lustre-MDT0001: new disk, initializing [ 444.926670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 444.962914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 444.968383] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 449.648824] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 460.129946] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 467.162860] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 475.810639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 476.180425] Lustre: lustre-OST0000: new disk, initializing [ 476.184805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 476.190754] Lustre: 17018:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 476.288526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 477.791979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 477.807089] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 477.967751] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 484.307801] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 497.973542] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 504.046737] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 504.292896] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 508.726848] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 508.907870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 514.021773] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 514.022395] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 514.031650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 514.031660] Lustre: Skipped 2 previous similar messages [ 514.038876] Lustre: Skipped 1 previous similar message [ 514.061017] LustreError: Skipped 1 previous similar message [ 516.840896] Lustre: server umount lustre-OST0000 complete [ 521.701481] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 521.709538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 529.376964] LustreError: 15061:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 529.419252] LustreError: 15061:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 529.824785] Lustre: server umount lustre-MDT0000 complete [ 533.473106] LustreError: 16194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 533.493113] LustreError: 16194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 534.917548] LustreError: 15041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423092 with bad export cookie 14197179033764647047 [ 534.921256] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 534.929485] LustreError: 15041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 535.646600] Lustre: server umount lustre-MDT0001 complete [ 543.385346] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:11:38 (1788423098) [ 552.383486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.312981] LustreError: 19286:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.464939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 559.868955] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 569.825633] LustreError: 19286:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.861576] LustreError: 19286:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 573.932563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.850119] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 589.870357] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 597.736575] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 605.705621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 614.436372] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 623.479851] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 633.823678] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 645.102599] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 645.115621] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 645.126149] Lustre: Skipped 3 previous similar messages [ 645.147848] Lustre: Skipped 6 previous similar messages [ 648.270661] Lustre: server umount lustre-MDT0000 complete [ 650.212147] LustreError: 20015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 650.258651] LustreError: 20015:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 652.567384] LustreError: 19268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423210 with bad export cookie 14197179033764647971 [ 652.581419] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 652.851772] Lustre: server umount lustre-MDT0001 complete [ 667.368831] Lustre: server umount lustre-OST0000 complete [ 691.046562] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 703.681372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 704.237738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 704.267053] Lustre: Skipped 2 previous similar messages [ 709.201751] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 714.732091] LustreError: 23671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 714.741425] LustreError: 23671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 718.123592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 723.143179] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 726.312398] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 733.155694] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 742.064064] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 753.171908] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 753.449564] Lustre: lustre-OST0001: new disk, initializing [ 753.459511] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 753.470943] Lustre: 25989:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 754.822819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 754.866946] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 755.083388] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 760.074149] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 770.187800] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 773.843682] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 785.895155] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 785.909097] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 785.914735] Lustre: Skipped 1 previous similar message [ 785.940226] Lustre: Skipped 3 previous similar messages [ 792.448167] Lustre: server umount lustre-MDT0000 complete [ 796.867515] LustreError: 24810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423354 with bad export cookie 14197179033764648874 [ 796.872751] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 796.879035] LustreError: 24810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 797.452048] Lustre: server umount lustre-MDT0001 complete [ 812.196872] Lustre: server umount lustre-OST0000 complete [ 826.448561] Lustre: server umount lustre-OST0001 complete [ 848.062308] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 860.759136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 861.136990] LustreError: 29010:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 861.168663] LustreError: 29010:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 861.292322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 861.315185] Lustre: Skipped 3 previous similar messages [ 867.636667] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 880.025538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 887.843759] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 892.617873] Lustre: 30160:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 900.489706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 906.643658] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 915.894149] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 924.648275] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 933.435670] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 937.733790] Lustre: 32042:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 947.169104] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 947.178661] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 947.195841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 947.203178] Lustre: Skipped 4 previous similar messages [ 947.244162] Lustre: Skipped 7 previous similar messages [ 951.687559] Lustre: server umount lustre-MDT0000 complete [ 956.909051] LustreError: 28992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423514 with bad export cookie 14197179033764650288 [ 956.932296] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 957.359372] Lustre: server umount lustre-MDT0001 complete [ 972.420974] Lustre: server umount lustre-OST0000 complete [ 974.175164] Lustre: 3612:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788423515/real 1788423515] req@ffff8bd584230380 x1875297035421056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788423531 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 977.390334] Lustre: 3615:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788423519/real 1788423519] req@ffff8bd482386d80 x1875297035421312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788423535 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 977.727757] Lustre: server umount lustre-OST0001 complete [ 990.636443] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:19:06 (1788423546) [ 1020.907327] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1032.044467] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1038.730419] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1049.771579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1055.277858] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1058.007558] Lustre: 35998:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1066.351624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1076.263539] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1087.357953] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1097.956748] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1109.472116] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1115.167652] Lustre: 37879:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1123.814458] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1123.838795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1123.846677] Lustre: Skipped 4 previous similar messages [ 1123.868689] Lustre: Skipped 3 previous similar messages [ 1128.625492] Lustre: server umount lustre-MDT0000 complete [ 1128.933445] LustreError: 34850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1128.984151] LustreError: 34850:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 24 previous similar messages [ 1133.225180] LustreError: 34830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423691 with bad export cookie 14197179033764651513 [ 1133.228351] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1133.246279] LustreError: 34830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 10 previous similar messages [ 1133.514824] Lustre: server umount lustre-MDT0001 complete [ 1149.180269] Lustre: server umount lustre-OST0000 complete [ 1149.219616] Lustre: 3612:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788423691/real 1788423691] req@ffff8bd5bd68f100 x1875297035483776/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788423707 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1154.016751] Lustre: 3615:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788423695/real 1788423695] req@ffff8bd5bffe3100 x1875297035484032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788423711 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1154.830380] Lustre: server umount lustre-OST0001 complete [ 1168.343606] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:22:03 (1788423723) [ 1188.416637] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1201.292923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1201.726186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1201.737986] Lustre: Skipped 7 previous similar messages [ 1206.487817] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1215.286824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.610790] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1224.096958] Lustre: 41171:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1232.066886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1240.472719] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1252.775864] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1260.410975] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1267.718707] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1271.425188] Lustre: 43049:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1272.504578] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1272.518917] Lustre: Skipped 1 previous similar message [ 1276.896683] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.924983] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1276.955489] Lustre: Skipped 2 previous similar messages [ 1278.919129] Lustre: server umount lustre-MDT0000 complete [ 1286.092311] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423843 with bad export cookie 14197179033764652738 [ 1286.094568] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1286.099806] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1292.874466] Lustre: server umount lustre-MDT0001 complete [ 1303.728386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1310.698979] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xc506831c802a6b7d [ 1310.718744] Lustre: mgc192.168.202.101@tcp: Connection restored to 0@lo (at 0@lo) [ 1316.231750] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1320.303183] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1326.076762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1331.706492] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1331.718698] Lustre: Skipped 1 previous similar message [ 1332.013506] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1340.052253] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1341.747866] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1348.432663] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1350.671819] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1357.292921] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1362.753737] Lustre: server umount lustre-MDT0000 complete [ 1367.646264] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788423925 with bad export cookie 14197179033764653949 [ 1367.682669] LustreError: 40004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1367.687578] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1368.490500] Lustre: server umount lustre-MDT0001 complete [ 1382.533444] Lustre: server umount lustre-OST0000 complete [ 1397.655352] Lustre: server umount lustre-OST0001 complete [ 1407.740738] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:26:03 (1788423963) [ 1428.316291] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1440.698846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1447.419804] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1458.616818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1467.736298] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1472.279239] Lustre: 48820:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1480.988297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1490.584838] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1500.745889] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1507.822075] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1518.010331] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1525.350681] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1546.211554] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1546.226687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1546.238697] Lustre: Skipped 9 previous similar messages [ 1546.244278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1546.253404] Lustre: Skipped 11 previous similar messages [ 1548.753872] Lustre: server umount lustre-MDT0000 complete [ 1554.354307] LustreError: 49205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788424112 with bad export cookie 14197179033764654866 [ 1554.360135] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1554.363541] LustreError: 49205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1555.241369] Lustre: server umount lustre-MDT0001 complete [ 1570.646072] Lustre: server umount lustre-OST0000 complete [ 1573.407837] Lustre: 3615:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788424115/real 1788424115] req@ffff8bd482594e00 x1875297035653376/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788424131 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1575.955546] Lustre: server umount lustre-OST0001 complete [ 1589.095821] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:29:04 (1788424144) [ 1592.712205] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1594.730575] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:29:10 (1788424150) [ 1608.505785] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 1620.350120] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1655.727758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1656.048435] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1656.062285] Lustre: Skipped 1 previous similar message [ 1656.092747] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1656.181573] Lustre: lustre-MDT0000: new disk, initializing [ 1656.359639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1662.650974] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1675.346651] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1685.263191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1685.764708] Lustre: lustre-OST0000: new disk, initializing [ 1685.772452] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1685.784330] Lustre: Skipped 1 previous similar message [ 1685.799387] Lustre: 54905:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1687.225175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1687.241149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1687.282619] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1693.046719] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1704.584623] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1707.242323] Lustre: DEBUG MARKER: create test files [ 1754.084548] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1756.039366] Lustre: server umount lustre-OST0000 complete [ 1760.817868] Lustre: server umount lustre-MDT0000 complete [ 1774.233878] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 1777.672378] Key type lgssc unregistered [ 1778.014229] LNet: 56540:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1778.020473] LNetError: 56540:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1779.054228] LNet: Removed LNI 192.168.202.101@tcp [ 1780.359199] Key type .llcrypt unregistered [ 1780.363782] Key type ._llcrypt unregistered [ 1815.955984] Key type ._llcrypt registered [ 1815.962175] Key type .llcrypt registered [ 1816.114833] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1817.938484] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1818.001708] alg: No test for adler32 (adler32-zlib) [ 1819.121935] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 1819.382991] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 1821.016892] Key type lgssc registered [ 1822.601669] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1834.025259] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1834.058777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1835.845453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1843.135463] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1853.826518] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1864.798994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1865.149284] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.622217] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 1880.509976] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1880.687918] Lustre: 59272:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1880.694860] Lustre: 59272:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1880.714932] Lustre: 59272:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1880.732219] Lustre: 59272:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1880.744655] Lustre: 59272:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1880.757948] Lustre: 59272:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1883.805944] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1890.652580] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1897.701220] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1906.145800] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1906.163223] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1906.177759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1911.270603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1912.208324] Lustre: server umount lustre-OST0000 complete [ 1916.599953] Lustre: server umount lustre-MDT0000 complete [ 1929.104832] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 1933.208437] Key type lgssc unregistered [ 1933.590043] LNet: 60884:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1933.598693] LNetError: 60884:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1934.680298] LNet: Removed LNI 192.168.202.101@tcp [ 1936.185779] Key type .llcrypt unregistered [ 1936.187510] Key type ._llcrypt unregistered [ 1979.397669] Key type ._llcrypt registered [ 1979.400384] Key type .llcrypt registered [ 1979.574684] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 1980.347585] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1980.447871] alg: No test for adler32 (adler32-zlib) [ 1981.641157] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 1981.852300] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 1983.535195] Key type lgssc registered [ 1984.560756] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1993.486923] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1993.519736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1994.918313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1999.343059] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2007.627263] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2015.554973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2015.913393] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2020.025209] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2020.111584] Lustre: 63716:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2020.122289] Lustre: 63716:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2020.131924] Lustre: 63716:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2020.140608] Lustre: 63716:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2020.151374] Lustre: 63716:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2020.162091] Lustre: 63716:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2022.536165] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2030.633669] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2036.379846] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2045.922407] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2045.937909] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2045.961109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2051.042373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2051.695718] Lustre: server umount lustre-OST0000 complete [ 2056.511412] Lustre: server umount lustre-MDT0000 complete [ 2072.832586] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 2077.769975] Key type lgssc unregistered [ 2078.688854] LNet: 65276:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2078.704358] LNetError: 65276:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2078.759552] LNet: Removed LNI 192.168.202.101@tcp [ 2080.250540] Key type .llcrypt unregistered [ 2080.253368] Key type ._llcrypt unregistered [ 2108.111700] Key type ._llcrypt registered [ 2108.116919] Key type .llcrypt registered [ 2108.230455] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 2126.570735] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 2128.184869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2128.235797] alg: No test for adler32 (adler32-zlib) [ 2129.330324] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 2129.607825] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 2131.295182] Key type lgssc registered [ 2132.928307] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2174.253104] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2174.281367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.577796] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2175.615390] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2175.687325] Lustre: lustre-MDT0000: new disk, initializing [ 2175.760137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2175.780660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2180.721807] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2193.016498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2193.169437] Lustre: 68429:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2193.303734] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2193.314532] Lustre: Skipped 1 previous similar message [ 2193.473198] Lustre: lustre-MDT0001: new disk, initializing [ 2193.616026] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2193.657892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2193.669482] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2199.071583] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2209.339799] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2216.014856] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2224.594404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2225.332647] Lustre: lustre-OST0000: new disk, initializing [ 2225.346644] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2225.376420] Lustre: 70396:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2225.619512] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2233.059338] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2234.031571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2234.045912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2234.182870] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2244.089654] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2249.438591] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2249.694237] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2254.606990] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2254.912118] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2259.943959] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2259.953190] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2259.959367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2259.985641] Lustre: Skipped 1 previous similar message [ 2263.307469] Lustre: server umount lustre-OST0000 complete [ 2270.176490] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2270.188968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2270.212040] Lustre: Skipped 1 previous similar message [ 2270.221637] Lustre: Skipped 2 previous similar messages [ 2274.107353] Lustre: server umount lustre-MDT0000 complete [ 2278.013924] LustreError: 68421:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788424835 with bad export cookie 8892988270762643129 [ 2278.062415] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2278.451967] Lustre: server umount lustre-MDT0001 complete [ 2288.270830] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:40:43 (1788424843) [ 2300.656765] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 2345.735442] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 2355.413536] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 2393.014392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2393.516484] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2393.556836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2393.717958] Lustre: lustre-MDT0000: new disk, initializing [ 2393.823202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2393.848599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2398.451526] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2409.233681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2409.367820] Lustre: 76419:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2409.506419] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2409.514808] Lustre: Skipped 1 previous similar message [ 2409.585400] Lustre: lustre-MDT0001: new disk, initializing [ 2409.697255] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2409.746443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2409.765282] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2414.127323] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2424.609983] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2431.436509] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2440.403166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2440.758782] Lustre: lustre-OST0000: new disk, initializing [ 2440.768976] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2440.787541] Lustre: 78385:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2440.924396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2447.970946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2447.990824] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2448.043171] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2448.166981] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2458.572414] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2464.407057] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2464.678093] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2470.244821] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2470.479398] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2473.958697] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2473.959211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2473.986927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2473.993807] LustreError: Skipped 2 previous similar messages [ 2474.022789] Lustre: Skipped 1 previous similar message [ 2479.087524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2484.195063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2484.202737] Lustre: Skipped 2 previous similar messages [ 2487.264991] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2487.397300] Lustre: server umount lustre-OST0000 complete [ 2491.880510] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2491.890505] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.904179] Lustre: Skipped 1 previous similar message [ 2494.432929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2494.438152] Lustre: Skipped 1 previous similar message [ 2497.598032] Lustre: server umount lustre-MDT0000 complete [ 2499.565972] LustreError: 76430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2499.593668] LustreError: 76430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2501.616862] LustreError: 76409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788425059 with bad export cookie 8892988270762644046 [ 2501.631422] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2501.644171] LustreError: 76409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2502.026861] Lustre: server umount lustre-MDT0001 complete [ 2510.997733] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:44:26 (1788425066) [ 2518.427318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2518.998767] LustreError: 80803:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2519.065430] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2523.466881] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2524.129164] LustreError: 80802:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2524.136282] LustreError: 80802:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2528.226811] LustreError: 80803:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2531.989242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2538.538949] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2547.779300] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2553.615670] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2560.493236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2560.837293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2560.840150] Lustre: Skipped 1 previous similar message [ 2566.417883] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2574.070569] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2578.482765] Lustre: 83471:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 37e574fd-7e28-4b44-b02a-13249e5f1aa3 at adminstrative request [ 2582.506984] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2582.528739] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2592.742748] Lustre: MGS: Client 37e574fd-7e28-4b44-b02a-13249e5f1aa3 (at 0@lo) reconnecting [ 2592.748913] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7b6a3db71685d5f1 to 0x7b6a3db71685d8c9 [ 2592.762347] Lustre: mgc192.168.202.101@tcp: Connection restored to 0@lo (at 0@lo) [ 2617.775563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2618.036849] Lustre: lustre-OST0001: new disk, initializing [ 2618.041955] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2618.047743] Lustre: 83867:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2624.295528] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2624.752776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2624.759808] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2624.899421] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2634.528848] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2640.356735] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2640.367445] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2640.388696] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2640.399558] Lustre: Skipped 2 previous similar messages [ 2640.422235] Lustre: Skipped 1 previous similar message [ 2643.713214] Lustre: server umount lustre-OST0001 complete [ 2645.485846] LustreError: 82624:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2645.501595] LustreError: 82624:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2649.057623] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2649.071237] LustreError: Skipped 1 previous similar message [ 2649.072669] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2655.714946] LustreError: 83507:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2655.747708] LustreError: 83507:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2661.343067] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2661.509721] Lustre: server umount lustre-OST0000 complete [ 2665.440725] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2665.449909] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2665.469598] Lustre: Skipped 1 previous similar message [ 2671.407948] Lustre: server umount lustre-MDT0000 complete [ 2675.120262] LustreError: 80783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788425232 with bad export cookie 8892988270762645705 [ 2675.133302] LustreError: 80783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2675.147145] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2675.476885] Lustre: server umount lustre-MDT0001 complete [ 2684.452501] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:47:20 (1788425240) [ 2685.936725] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2688.157459] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:47:23 (1788425243) [ 2689.826871] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2691.810205] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:47:27 (1788425247) [ 2693.633801] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2695.662137] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:47:31 (1788425251) [ 2697.487765] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2699.403268] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:47:35 (1788425255) [ 2701.336553] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2708.120161] loop: module loaded [ 2708.159480] loop0: detected capacity change from 0 to 4294967296 [ 2708.263531] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2709.880398] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2714.408572] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2717.802193] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2746.305134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2746.365825] LustreError: 87990:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2746.376449] LustreError: 87990:0:(obd_mount.c:337:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2746.386822] LustreError: 87990:0:(tgt_mount.c:2721:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2746.392596] LustreError: 87990:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2767.353313] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 2779.120467] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 2817.828560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.123828] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2818.207884] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2818.324836] Lustre: lustre-MDT0000: new disk, initializing [ 2818.400243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2818.409553] Lustre: Skipped 1 previous similar message [ 2818.424252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2822.825726] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2833.479412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2833.563546] Lustre: 90856:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2833.796488] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2838.054978] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2848.693578] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2853.904579] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2861.246770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2861.586283] Lustre: 92828:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2868.417347] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2868.875524] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2869.018546] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2879.639411] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2884.776081] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2884.974231] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2889.258441] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2889.462516] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2894.821080] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2894.822891] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2894.824676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2894.824684] Lustre: Skipped 11 previous similar messages [ 2894.840841] Lustre: Skipped 2 previous similar messages [ 2894.868439] LustreError: Skipped 1 previous similar message [ 2897.493984] Lustre: server umount lustre-OST0000 complete [ 2916.327081] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2916.439270] Lustre: server umount lustre-MDT0000 complete [ 2919.340111] LustreError: 90849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788425477 with bad export cookie 8892988270762646433 [ 2919.349052] LustreError: 90849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2919.351200] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2919.726094] Lustre: server umount lustre-MDT0001 complete [ 2928.348464] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 04:51:24 (1788425484) [ 2934.614711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2934.962589] LustreError: 95242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2934.987189] LustreError: 95242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2939.788612] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2947.080631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2947.640252] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2947.645791] Lustre: Skipped 3 previous similar messages [ 2951.790483] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2958.428801] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2963.798440] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2971.125577] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2978.296622] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 2988.052418] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2994.151596] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2994.493585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3000.237643] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3000.654709] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3007.960947] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3008.230524] Lustre: lustre-OST0001: new disk, initializing [ 3008.232723] Lustre: Skipped 2 previous similar messages [ 3008.247896] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3008.257347] Lustre: Skipped 3 previous similar messages [ 3008.265235] Lustre: 98614:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3015.493399] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3016.934128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3016.946875] Lustre: Skipped 2 previous similar messages [ 3016.949369] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3017.105805] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3024.414899] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3030.615258] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3030.921724] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3037.045773] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3037.274467] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3058.441909] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3083.690321] Lustre: Failing over lustre-MDT0000 [ 3083.749667] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3083.762448] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3083.768243] Lustre: Skipped 5 previous similar messages [ 3083.788179] Lustre: Skipped 10 previous similar messages [ 3084.053800] Lustre: server umount lustre-MDT0000 complete [ 3085.734701] LustreError: 97082:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.1@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3085.755270] LustreError: 97082:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3094.008665] LustreError: mgc192.168.202.101@tcp: operation ldlm_enqueue to node 0@lo failed: rc = -107 [ 3094.016404] LustreError: 98613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788425651 with bad export cookie 8892988270762647357 [ 3094.019234] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3094.025784] Lustre: 100833:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3094.027934] Lustre: 95234:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -107 [ 3094.037632] LustreError: 98613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3099.620458] Lustre: 100831:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3099.633063] Lustre: 100831:0:(ldlm_lib.c:1251:target_handle_connect()) Skipped 1 previous similar message [ 3104.740239] Lustre: 100833:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3107.335265] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3107.339740] LDISKFS-fs (dm-0): recovery complete [ 3107.344778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3107.457433] Lustre: 95234:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -2 [ 3107.754591] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3110.426644] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3110.430770] Lustre: 98622:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3111.315048] Lustre: 95237:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3112.022920] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3112.932465] Lustre: 95242:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3112.946945] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3113.054154] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3113.376431] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3113.727413] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3113.741958] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 1 previous similar message [ 3114.063149] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3114.077887] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 1 previous similar message [ 3114.730774] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3114.741790] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 2 previous similar messages [ 3115.064305] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3115.072812] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 2 previous similar messages [ 3115.092515] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3115.109849] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) Skipped 10 previous similar messages [ 3117.041923] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3117.053350] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 6 previous similar messages [ 3117.381335] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3117.387378] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 6 previous similar messages [ 3119.352387] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3119.361289] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) Skipped 12 previous similar messages [ 3121.305538] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3121.324914] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 12 previous similar messages [ 3121.647191] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3121.652166] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 12 previous similar messages [ 3127.394913] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3127.406894] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) Skipped 24 previous similar messages [ 3129.322089] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3129.333311] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 24 previous similar messages [ 3129.655136] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3129.666528] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 24 previous similar messages [ 3143.640320] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3143.670632] Lustre: 101469:0:(ldlm_lib.c:2127:extend_recovery_timer()) Skipped 50 previous similar messages [ 3145.603904] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3145.622877] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 50 previous similar messages [ 3145.952876] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3145.973846] LustreError: 101469:0:(ldlm_lib.c:2740:replay_request_or_update()) Skipped 50 previous similar messages [ 3170.112753] Lustre: 101469:0:(ldlm_lib.c:2436:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3170.125599] Lustre: 101469:0:(ldlm_lib.c:2446:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3170.130140] Lustre: 101469:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d867e773-2e29-435a-82cd-dc85499b0f07@192.168.202.1@tcp [ 3170.136692] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3170.140776] Lustre: 101469:0:(ldlm_lib.c:1956:abort_req_replay_queue()) @@@ aborted: req@000000006aeb70e8 x1875299184743808/t0(8589934775) o101->d867e773-2e29-435a-82cd-dc85499b0f07@192.168.202.1@tcp:613/0 lens 592/0 e 0 to 0 dl 1788425738 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3170.173334] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3170.174928] Lustre: lustre-MDT0000: Denying connection for new client d867e773-2e29-435a-82cd-dc85499b0f07 (at 192.168.202.1@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3170.193289] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3170.219501] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3170.222160] Lustre: Skipped 2 previous similar messages [ 3170.223709] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3170.260614] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3170.261165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3189.432153] Lustre: server umount lustre-OST0000 complete [ 3189.745487] LustreError: 97064:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3189.772356] LustreError: 97064:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 29 previous similar messages [ 3199.247918] Lustre: server umount lustre-OST0001 complete [ 3214.305334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3214.307434] Lustre: Skipped 8 previous similar messages [ 3217.376616] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3217.574308] Lustre: server umount lustre-MDT0000 complete [ 3221.418177] LustreError: 95223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788425779 with bad export cookie 8892988270762697911 [ 3221.429432] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3221.430132] LustreError: 95223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3221.755561] Lustre: server umount lustre-MDT0001 complete [ 3229.654206] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 04:56:25 (1788425785) [ 3236.463896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3236.480680] Lustre: *** cfs_fail_loc=197, val=0*** [ 3236.576303] Lustre: 103455:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3241.954849] Lustre: 103458:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3252.063577] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3252.195736] Lustre: 103458:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3252.213959] Lustre: 103458:0:(ldlm_lib.c:1251:target_handle_connect()) Skipped 1 previous similar message [ 3257.183538] LustreError: 103443:0:(mgc_request.c:317:do_config_log_add()) mgc192.168.202.101@tcp: failed processing log, type 4: rc = -110 [ 3262.330980] Lustre: *** cfs_fail_loc=197, val=0*** [ 3262.333442] Lustre: Skipped 66 previous similar messages [ 3267.487952] Lustre: 103443:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -110 [ 3272.677340] Lustre: 103458:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3272.691746] Lustre: 103458:0:(ldlm_lib.c:1251:target_handle_connect()) Skipped 3 previous similar messages [ 3277.919460] Lustre: 103460:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -110 [ 3288.039264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3288.053801] Lustre: Skipped 3 previous similar messages [ 3288.159646] LustreError: 103460:0:(mgc_request.c:718:do_requeue()) failed processing log: -110 [ 3294.773965] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3303.045944] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3307.488858] Lustre: 103455:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 3307.512901] Lustre: 103455:0:(ldlm_lib.c:1251:target_handle_connect()) Skipped 6 previous similar messages [ 3308.703747] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3309.177106] Lustre: server umount lustre-OST0000 complete [ 3317.869223] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 04:57:53 (1788425873) [ 3338.922671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3339.544996] LustreError: 105075:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3339.565732] LustreError: 105075:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3344.169992] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3354.627886] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3358.514069] LustreError: 105106:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3358.524052] LustreError: 105106:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3358.527961] LustreError: 105106:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3358.751758] Lustre: server umount lustre-MDT0000 complete [ 3390.327841] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 3401.219410] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 3439.429783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3439.703146] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3439.726303] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3439.805196] Lustre: lustre-MDT0000: new disk, initializing [ 3439.932211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3444.566765] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3455.203931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3455.332786] Lustre: 109279:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3455.735855] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3459.690324] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3469.463483] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3474.718739] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3482.136611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3482.505057] Lustre: 111249:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3488.101580] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3489.741845] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3500.504915] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3506.347501] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3506.553423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3511.651326] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3511.837737] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3513.829529] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3513.839067] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3513.843119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3513.874725] Lustre: Skipped 7 previous similar messages [ 3519.932324] Lustre: server umount lustre-OST0000 complete [ 3530.573298] Lustre: server umount lustre-MDT0000 complete [ 3536.299199] LustreError: 109273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788426094 with bad export cookie 8892988270762708719 [ 3536.317431] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3536.323564] LustreError: 109273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3537.115525] Lustre: server umount lustre-MDT0001 complete [ 3551.887587] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:01:46 (1788426106) [ 3560.990750] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 3563.648487] Key type lgssc unregistered [ 3563.931450] LNet: 113753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3563.948563] LNetError: 113753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3563.986350] LNet: Removed LNI 192.168.202.101@tcp [ 3564.853798] Key type .llcrypt unregistered [ 3564.861782] Key type ._llcrypt unregistered [ 3581.626209] Key type ._llcrypt registered [ 3581.630518] Key type .llcrypt registered [ 3581.738408] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 3595.473765] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 3596.725236] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3596.803697] alg: No test for adler32 (adler32-zlib) [ 3598.026174] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 3598.403765] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 3600.111293] Key type lgssc registered [ 3601.507577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3645.160022] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3645.192654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3646.506956] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3646.548468] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3646.622223] Lustre: lustre-MDT0000: new disk, initializing [ 3646.735053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3646.753561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3651.346413] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3662.724994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3663.138499] Lustre: lustre-OST0000: new disk, initializing [ 3663.152641] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3663.156651] Lustre: Skipped 1 previous similar message [ 3663.164385] Lustre: 117924:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3663.263515] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3665.767588] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3665.795452] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3665.864230] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3670.730909] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3682.098074] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3682.272658] Lustre: lustre-OST0001: new disk, initializing [ 3682.276582] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3682.282618] Lustre: 118788:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3682.361457] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3689.153051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3689.163981] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3689.193322] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3690.208701] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3699.682634] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3699.689977] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3699.706434] Lustre: Skipped 1 previous similar message [ 3699.725413] Lustre: Skipped 1 previous similar message [ 3704.802191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3704.810532] Lustre: Skipped 1 previous similar message [ 3706.160210] Lustre: server umount lustre-MDT0000 complete [ 3714.001871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3719.010896] LustreError: 117111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788426276 with bad export cookie 11209622272999694043 [ 3719.020266] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3719.038066] LustreError: 117111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3719.208523] Lustre: server umount lustre-OST0000 complete [ 3723.481078] Lustre: server umount lustre-OST0001 complete [ 3732.746422] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:04:48 (1788426288) [ 3756.609197] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:05:12 (1788426312) [ 3770.529532] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 3799.722853] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:05:55 (1788426355) [ 3843.672597] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 3859.069546] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 3903.409872] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3903.847837] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3903.919179] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3904.050819] Lustre: lustre-MDT0000: new disk, initializing [ 3904.200274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3904.241624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3908.996838] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3919.759240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3919.860778] Lustre: 126569:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3919.948750] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3919.951840] Lustre: Skipped 1 previous similar message [ 3920.022975] Lustre: lustre-MDT0001: new disk, initializing [ 3920.124662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3920.162740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3920.178444] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3924.592770] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3934.555736] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3941.194957] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3949.081967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3949.468254] Lustre: lustre-OST0000: new disk, initializing [ 3949.475501] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3949.485553] Lustre: 128539:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3949.580160] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3956.431636] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 3958.368799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3958.381109] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3958.488876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3966.408284] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3971.092832] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3971.380420] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3976.184626] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3976.363319] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3979.233967] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3979.236442] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3979.237533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3979.262733] LustreError: Skipped 1 previous similar message [ 3979.293499] Lustre: Skipped 1 previous similar message [ 3984.407777] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3984.413191] Lustre: Skipped 2 previous similar messages [ 3984.788136] Lustre: server umount lustre-OST0000 complete [ 3992.033046] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3992.037562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3992.044348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3994.607515] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4004.321498] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4005.035514] Lustre: server umount lustre-MDT0000 complete [ 4009.731593] LustreError: 126562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788426567 with bad export cookie 11209622272999695352 [ 4009.733759] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4009.766386] LustreError: 126562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4010.462407] Lustre: server umount lustre-MDT0001 complete [ 4024.501434] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:09:39 (1788426579) [ 4033.666873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4034.485876] LustreError: 130956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4034.516788] LustreError: 130956:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4034.673618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4040.165219] LustreError: 130957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4040.782792] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4044.282895] LustreError: 130956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4049.378254] LustreError: 130957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4050.157288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4055.050683] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4065.133920] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4072.009402] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4079.709895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4080.046080] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4080.055843] Lustre: Skipped 1 previous similar message [ 4087.107192] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4096.253320] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4101.580941] Lustre: *** cfs_fail_loc=159, val=0*** [ 4101.586654] Lustre: Skipped 5 previous similar messages [ 4116.976692] Lustre: lustre-MDT0000: Client c83d43f2-a8bc-4a04-80a5-23dc37157f4e (at 192.168.202.1@tcp) reconnecting [ 4118.424858] Lustre: *** cfs_fail_loc=159, val=0*** [ 4133.891298] Lustre: lustre-MDT0000: Client c83d43f2-a8bc-4a04-80a5-23dc37157f4e (at 192.168.202.1@tcp) reconnecting [ 4141.540516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.560089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4141.562926] Lustre: Skipped 4 previous similar messages [ 4141.564725] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4145.625781] Lustre: server umount lustre-OST0000 complete [ 4151.780690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4151.807859] Lustre: Skipped 2 previous similar messages [ 4156.518694] Lustre: server umount lustre-MDT0000 complete [ 4156.907331] LustreError: 131686:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4156.927117] LustreError: 131686:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4161.132532] LustreError: 130936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788426718 with bad export cookie 11209622272999696276 [ 4161.139978] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4161.151673] LustreError: 130936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4161.817722] Lustre: server umount lustre-MDT0001 complete [ 4175.714188] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 4178.432697] Key type lgssc unregistered [ 4178.810569] LNet: 134733:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4178.818292] LNetError: 134733:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4178.839799] LNet: Removed LNI 192.168.202.101@tcp [ 4179.722977] Key type .llcrypt unregistered [ 4179.726053] Key type ._llcrypt unregistered [ 4213.272858] Key type ._llcrypt registered [ 4213.276460] Key type .llcrypt registered [ 4213.396191] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 4215.687137] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4215.716868] alg: No test for adler32 (adler32-zlib) [ 4216.846787] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 4217.230945] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 4219.019719] Key type lgssc registered [ 4220.766377] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4228.964536] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4228.987079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4230.355600] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4234.895883] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4240.874065] LustreError: 136144:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4240.919786] LustreError: 136144:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4245.987487] LustreError: 136145:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4246.792737] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4247.666886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4255.249833] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4269.005552] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4278.653206] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4286.913397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4287.331283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4294.939887] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4300.727363] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4303.617394] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4314.211831] Lustre: *** cfs_fail_loc=159, val=0*** [ 4330.445230] Lustre: lustre-MDT0000: Client 87471186-a4b0-40a1-b445-1fb12d888d2e (at 192.168.202.1@tcp) reconnecting [ 4333.538811] Lustre: *** cfs_fail_loc=159, val=0*** [ 4333.578612] Lustre: Skipped 1 previous similar message [ 4348.880840] Lustre: lustre-MDT0001: Client 87471186-a4b0-40a1-b445-1fb12d888d2e (at 192.168.202.1@tcp) reconnecting [ 4350.916723] Lustre: *** cfs_fail_loc=159, val=0*** [ 4350.925378] Lustre: Skipped 3 previous similar messages [ 4367.363953] Lustre: lustre-MDT0001: Client 87471186-a4b0-40a1-b445-1fb12d888d2e (at 192.168.202.1@tcp) reconnecting [ 4374.558614] Lustre: 139186:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4378.534124] Lustre: *** cfs_fail_loc=159, val=0*** [ 4378.535830] Lustre: Skipped 2 previous similar messages [ 4393.992988] Lustre: lustre-MDT0000: Client 89d74332-29ac-4788-8f3c-c944fcc55cff (at 192.168.202.1@tcp) reconnecting [ 4397.308422] Lustre: *** cfs_fail_loc=159, val=0*** [ 4397.314378] Lustre: Skipped 12 previous similar messages [ 4412.897675] Lustre: lustre-MDT0000: Client 89d74332-29ac-4788-8f3c-c944fcc55cff (at 192.168.202.1@tcp) reconnecting [ 4414.564459] Lustre: 139480:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4435.938272] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4435.966690] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4435.984411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4439.010084] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4439.014863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4439.060546] Lustre: Skipped 1 previous similar message [ 4440.685548] Lustre: server umount lustre-OST0000 complete [ 4447.200184] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4447.211495] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4447.245202] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4449.252242] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4449.253824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4449.283312] Lustre: Skipped 1 previous similar message [ 4453.168217] Lustre: server umount lustre-MDT0000 complete [ 4454.368321] LustreError: 136144:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4454.397567] LustreError: 136144:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4457.756439] LustreError: 136127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788427015 with bad export cookie 17134427902387495180 [ 4457.758087] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4457.773364] LustreError: 136127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4458.583829] Lustre: server umount lustre-MDT0001 complete [ 4473.995250] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 4477.094652] Key type lgssc unregistered [ 4477.411122] LNet: 140557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4477.417924] LNetError: 140557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4477.436682] LNet: Removed LNI 192.168.202.101@tcp [ 4478.399780] Key type .llcrypt unregistered [ 4478.402956] Key type ._llcrypt unregistered [ 4510.893553] Key type ._llcrypt registered [ 4510.896221] Key type .llcrypt registered [ 4510.993288] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 4511.845237] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4511.877484] alg: No test for adler32 (adler32-zlib) [ 4512.879912] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 4513.137709] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 4514.823229] Key type lgssc registered [ 4516.316589] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4524.801711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4524.829852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4526.465904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4531.824549] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4536.818603] LustreError: 141971:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4536.835063] LustreError: 141971:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4541.441901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4541.907964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4546.619799] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4555.495453] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4563.795905] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4572.338996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4572.854888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4582.275620] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4588.237940] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4591.633792] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4596.484748] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4603.139573] Lustre: 144553:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4607.423204] Lustre: 144602:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4609.293938] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4610.404849] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4615.463209] Lustre: 144759:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4616.723811] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4617.747903] Lustre: 144857:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4624.354264] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4624.363736] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4624.383979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4627.425810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4627.434827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4627.443493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.474209] Lustre: Skipped 1 previous similar message [ 4628.932294] Lustre: server umount lustre-OST0000 complete [ 4634.092326] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4634.108189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4634.147532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4637.665807] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4637.666659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4637.710653] Lustre: Skipped 1 previous similar message [ 4640.396250] Lustre: server umount lustre-MDT0000 complete [ 4642.787196] LustreError: 141970:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4642.815154] LustreError: 141970:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4644.922323] LustreError: 144964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788427202 with bad export cookie 9587790042436826707 [ 4644.933335] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4644.939484] LustreError: 144964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4645.251038] Lustre: server umount lustre-MDT0001 complete [ 4659.419838] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 4663.701411] Key type lgssc unregistered [ 4664.022426] LNet: 145915:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4664.036460] LNetError: 145915:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4664.051817] LNet: Removed LNI 192.168.202.101@tcp [ 4664.895221] Key type .llcrypt unregistered [ 4664.897057] Key type ._llcrypt unregistered [ 4695.256211] Key type ._llcrypt registered [ 4695.258482] Key type .llcrypt registered [ 4695.318253] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 4696.396653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4696.763583] alg: No test for adler32 (adler32-zlib) [ 4698.363777] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 4698.688337] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 4700.383200] Key type lgssc registered [ 4702.704039] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4713.660060] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4713.706765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4715.351320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4720.880702] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4725.730500] LustreError: 147328:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4725.765074] LustreError: 147328:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4730.854045] LustreError: 147329:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4735.589832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4735.835355] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4742.064300] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4755.237974] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4764.505851] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4772.327757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4772.722698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4779.247872] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4780.915444] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4786.033276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4788.885829] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4795.104580] Lustre: *** cfs_fail_loc=159, val=0*** [ 4810.731213] Lustre: lustre-MDT0000: Client ea7cff00-6101-4ae6-8c8d-d8de08f9150e (at 192.168.202.1@tcp) reconnecting [ 4816.867487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4816.871881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4816.886432] LustreError: Skipped 1 previous similar message [ 4816.900415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4821.996719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4822.015771] Lustre: Skipped 2 previous similar messages [ 4822.891545] Lustre: server umount lustre-OST0000 complete [ 4828.128841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4828.135034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4828.142430] Lustre: Skipped 1 previous similar message [ 4828.147056] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4832.225221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4832.226325] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4835.708023] Lustre: server umount lustre-MDT0000 complete [ 4837.350037] LustreError: 149181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4837.373231] LustreError: 149181:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4840.123682] LustreError: 147308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788427397 with bad export cookie 15121389897013088123 [ 4840.129811] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4840.130381] LustreError: 147308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4840.566118] Lustre: server umount lustre-MDT0001 complete [ 4856.388874] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 4860.549539] Key type lgssc unregistered [ 4861.026725] LNet: 150999:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4861.041073] LNetError: 150999:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4861.058292] LNet: Removed LNI 192.168.202.101@tcp [ 4862.571796] Key type .llcrypt unregistered [ 4862.574453] Key type ._llcrypt unregistered [ 4891.852522] Key type ._llcrypt registered [ 4891.857282] Key type .llcrypt registered [ 4892.070555] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 4893.536497] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4893.603249] alg: No test for adler32 (adler32-zlib) [ 4894.856446] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 4895.124586] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 4896.897586] Key type lgssc registered [ 4898.289091] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4909.814189] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4909.844494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4911.695942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4917.486400] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4921.832738] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4921.854700] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4926.951381] LustreError: 152412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4929.198951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4929.726892] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4934.834454] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4943.718859] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4951.839973] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4959.538762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4960.096876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4967.314804] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 4974.463422] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4976.711587] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4981.626114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5002.223967] Lustre: lustre-OST0000: evicting 76b84d0d-61d1-473a-9b11-4abb5f55f420 (at 192.168.202.1@tcp) by administrative request [ 5002.246693] Lustre: Skipped 1 previous similar message [ 5032.938789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5032.945162] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5032.945778] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5032.952925] Lustre: Skipped 1 previous similar message [ 5038.050096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5038.062646] Lustre: Skipped 2 previous similar messages [ 5038.893671] Lustre: server umount lustre-OST0000 complete [ 5043.170318] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5043.178761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5043.190034] Lustre: Skipped 1 previous similar message [ 5048.288823] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5048.297246] Lustre: Skipped 2 previous similar messages [ 5049.640406] Lustre: server umount lustre-MDT0000 complete [ 5053.409860] LustreError: 152412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5053.440438] LustreError: 152412:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5053.583221] LustreError: 152393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788427611 with bad export cookie 15865200486861930363 [ 5053.585064] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5053.597558] LustreError: 152393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5054.036983] Lustre: server umount lustre-MDT0001 complete [ 5067.541397] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 5070.828957] Key type lgssc unregistered [ 5071.123553] LNet: 156484:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5071.136707] LNetError: 156484:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5071.159453] LNet: Removed LNI 192.168.202.101@tcp [ 5072.133891] Key type .llcrypt unregistered [ 5072.136925] Key type ._llcrypt unregistered [ 5171.387715] Key type ._llcrypt registered [ 5171.393858] Key type .llcrypt registered [ 5171.543869] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 5172.852544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5172.896279] alg: No test for adler32 (adler32-zlib) [ 5174.316036] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 5174.801510] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 5176.503299] Key type lgssc registered [ 5178.122521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5188.944674] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5189.000239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5190.777842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5195.784445] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5200.363126] LustreError: 158973:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5200.401914] LustreError: 158973:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5205.488819] LustreError: 158974:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5207.811958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5208.507871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5214.598908] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5223.091188] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5229.178238] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5237.129491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5237.619714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5244.790840] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5253.471213] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5254.017649] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5262.200957] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5272.547575] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5272.561254] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5272.571803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5277.666799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5277.679935] Lustre: Skipped 2 previous similar messages [ 5279.020499] Lustre: server umount lustre-OST0000 complete [ 5285.344754] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5285.355391] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5285.373468] Lustre: Skipped 1 previous similar message [ 5285.376893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5287.905653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5287.935100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5290.267451] Lustre: server umount lustre-MDT0000 complete [ 5293.024769] LustreError: 158973:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5293.053898] LustreError: 158973:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5294.836602] LustreError: 158955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788427852 with bad export cookie 1495778057554705467 [ 5294.840847] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5294.848128] LustreError: 158955:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5295.284391] Lustre: server umount lustre-MDT0001 complete [ 5313.756676] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 5317.233816] Key type lgssc unregistered [ 5317.480836] LNet: 162560:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5317.486635] LNetError: 162560:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5318.502694] LNet: Removed LNI 192.168.202.101@tcp [ 5319.431765] Key type .llcrypt unregistered [ 5319.435165] Key type ._llcrypt unregistered [ 5352.120854] Key type ._llcrypt registered [ 5352.124894] Key type .llcrypt registered [ 5352.236444] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 5353.607427] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5353.635542] alg: No test for adler32 (adler32-zlib) [ 5355.158536] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 5355.591962] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 5357.392390] Key type lgssc registered [ 5359.684220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5380.640216] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:32:16 (1788427936) [ 5400.823638] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 5418.270868] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 5463.225967] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5463.285200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5464.538990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5464.582345] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5464.687568] Lustre: lustre-MDT0000: new disk, initializing [ 5464.805048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5464.831090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5471.611114] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5484.383783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5484.510523] Lustre: 166847:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5484.608269] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5484.611912] Lustre: Skipped 1 previous similar message [ 5484.680409] Lustre: lustre-MDT0001: new disk, initializing [ 5484.727112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5484.770195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5484.786555] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5490.440974] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5502.370370] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5510.558879] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5518.767494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5519.401340] Lustre: lustre-OST0000: new disk, initializing [ 5519.412204] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5519.436811] Lustre: 168814:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5519.678176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5527.245097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5527.260304] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5527.489184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5530.067595] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5542.911808] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5558.242833] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5558.251232] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5558.260931] LustreError: Skipped 1 previous similar message [ 5558.261965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5558.303150] Lustre: Skipped 1 previous similar message [ 5560.663502] Lustre: server umount lustre-OST0000 complete [ 5566.944890] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5566.958939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5566.974330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5566.988662] Lustre: Skipped 1 previous similar message [ 5568.480634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5568.483593] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5571.771500] Lustre: server umount lustre-MDT0000 complete [ 5575.733855] LustreError: 166840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788428133 with bad export cookie 14993006305785670990 [ 5575.737244] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5575.743489] LustreError: 166840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5576.161102] Lustre: server umount lustre-MDT0001 complete [ 5588.090454] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 5590.759151] Key type lgssc unregistered [ 5591.106737] LNet: 170722:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5591.112246] LNetError: 170722:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5592.169944] LNet: Removed LNI 192.168.202.101@tcp [ 5593.319614] Key type .llcrypt unregistered [ 5593.324658] Key type ._llcrypt unregistered [ 5623.832654] Key type ._llcrypt registered [ 5623.835528] Key type .llcrypt registered [ 5623.956277] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 5625.205277] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5625.330179] alg: No test for adler32 (adler32-zlib) [ 5626.525701] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 5626.958623] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 5628.623307] Key type lgssc registered [ 5630.000651] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5639.773599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5639.803641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5641.424779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5646.795824] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5651.937749] LustreError: 172136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5651.953598] LustreError: 172136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5656.484746] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5656.795818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5662.838089] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5671.368324] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5678.325513] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5686.174040] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5686.600347] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5687.626048] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5694.491401] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 5705.168731] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5713.165884] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5713.193740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5713.208517] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5713.234724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.256802] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5714.251869] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5714.307441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5714.328292] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5714.354123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.374765] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5715.451063] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5715.471917] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5715.483231] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5715.490719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.515234] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5716.836931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.029094] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5718.052775] Lustre: Skipped 1 previous similar message [ 5718.095297] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5718.111701] Lustre: Skipped 1 previous similar message [ 5718.116558] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5718.126828] Lustre: Skipped 1 previous similar message [ 5718.132519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.140959] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5718.143784] Lustre: Skipped 1 previous similar message [ 5719.446266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.551509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.512766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.187309] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5722.201749] Lustre: Skipped 3 previous similar messages [ 5722.210733] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5722.217829] Lustre: Skipped 3 previous similar messages [ 5722.221463] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5722.224470] Lustre: Skipped 3 previous similar messages [ 5722.226271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.230082] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5722.232996] Lustre: Skipped 3 previous similar messages [ 5722.940752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.513331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.591978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.744900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.044903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.057661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.019672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.141954] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5731.149616] Lustre: Skipped 7 previous similar messages [ 5731.180467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5731.203227] Lustre: Skipped 7 previous similar messages [ 5731.215699] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5731.225865] Lustre: Skipped 7 previous similar messages [ 5731.237409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.260001] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5731.268114] Lustre: Skipped 7 previous similar messages [ 5732.458061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.815214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.903898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.524534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.200910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.607878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.885825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.080274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.462874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.419994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.438456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.311261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.568849] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5747.585058] Lustre: Skipped 12 previous similar messages [ 5747.615288] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5747.629059] Lustre: Skipped 12 previous similar messages [ 5747.632823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5747.637455] Lustre: Skipped 12 previous similar messages [ 5747.640452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.646580] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5747.649522] Lustre: Skipped 12 previous similar messages [ 5748.887724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.161881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.465175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.800935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.489717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.929991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.346325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.942996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.431238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.480482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.566245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.954710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.489380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.899390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.940492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.096378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.836828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.844136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.155766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.777144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.951047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.440794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.431626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.489803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.312031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.265403] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5780.279033] Lustre: Skipped 25 previous similar messages [ 5780.312588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5780.338579] Lustre: Skipped 25 previous similar messages [ 5780.352179] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5780.357752] Lustre: Skipped 25 previous similar messages [ 5780.363993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.371109] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5780.376382] Lustre: Skipped 25 previous similar messages [ 5781.619706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.409635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.349324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.556251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.551856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.449305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.744646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.463455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.841307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.475710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.644363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.573829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.462091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.470996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.534869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.630070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.847828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.798060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.002483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.006108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.146854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.325347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.342424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.230357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.089069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.213058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.057858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.033553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.101195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.066016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.229895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.274767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.031532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.041934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.002162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.898808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.036182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.130993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.306223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.302957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.478943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.727834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.330726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.605507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.906677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.959940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.967362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.196503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.257828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.038161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.992948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.160433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.208628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.647874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.856861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.748650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.908598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.890161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.164394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.213634] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5845.225950] Lustre: Skipped 59 previous similar messages [ 5845.253652] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5845.267858] Lustre: Skipped 59 previous similar messages [ 5845.278964] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5845.292429] Lustre: Skipped 59 previous similar messages [ 5845.305725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.331852] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5845.354787] Lustre: Skipped 59 previous similar messages [ 5846.677325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.646886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.687139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.101785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.316979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.623599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.799312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.909769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.945635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.991870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.060100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.429624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.627289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.962143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.197295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.301206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.718125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.172352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.417272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.576299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.802686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.798765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.047392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.006838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.173412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.170446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.113859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.108952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.122850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.137686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.996849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.260819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.284424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.212516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.293297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.398144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.572162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.761767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.798039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.641345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.595141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.660456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.847213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.017223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.054984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.319912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.410668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.646929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.807452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.029640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.408228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.698871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.811089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.999918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.926536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.017154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.412533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.759725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.109988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.007939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.246824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.203742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.389233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.417183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.405342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.510416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.842212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.885639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.720807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.750400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.007281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.447681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.806329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.188789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.774549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.506973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.918752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.239915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.718940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.764504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.888236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.386373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.924768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.760661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.448126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.447037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.712404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.934229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.748401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.184666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.394689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.590883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.110670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.271680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.105887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.528758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.736405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.954698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.818206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.174975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.089409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.063951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.310810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.978733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.422518] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5973.432746] Lustre: Skipped 104 previous similar messages [ 5973.458706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5973.478743] Lustre: Skipped 104 previous similar messages [ 5973.493185] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5973.498205] Lustre: Skipped 104 previous similar messages [ 5973.501537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.522231] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5973.529484] Lustre: Skipped 104 previous similar messages [ 5974.975715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.890704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.948583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.725453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.693760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.753165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.803684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.943187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.958565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.029567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.108054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.248228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.183782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.099238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.009164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.021920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.933791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.957635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.097483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.422394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.671536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.174807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.190927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.076382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.162937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.095529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.147136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.070123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.051457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.030549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.284868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.283954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.436088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.531982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.701019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.642871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.787188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.947914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.218051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.431770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.469424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.389866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.309957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.307589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.285363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.557950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.680829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.575836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.833382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.994859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.136714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.406230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.924468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.904969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.925824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.956740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.192821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.298054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.240893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.351644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.432076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.271769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.440227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.660851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.639983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.097546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.122678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.099774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.002423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.016431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.078251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.899537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.991648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.950981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.982446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.851805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.880744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.025144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.211947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.456870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.756899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.859754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.813764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.952058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.793149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.894334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.802863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.817917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.710679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.686624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.889410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.753833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.727168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.374053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.412252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.325693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.130607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.969245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.797771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.791512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.859397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.786697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.770545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.753846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.864273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.687367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.728828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.660207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.555322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.529578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.598673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.527332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.581778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.370167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.455889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.484992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.501782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.388146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.460360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.362427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.280505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.208404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.298494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.231078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.220082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.137043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.198313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.112480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.114904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.994258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.981922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.911324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.964656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.986784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.944132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.728334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.594961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.446619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.349212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.332582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.398622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.363849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.327568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.229564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.188307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.070541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.800395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.647720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.492969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.377727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.509883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.723325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.476934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.341167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.261521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.252303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.432645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.651929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.939980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.978239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.969674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.843200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.059271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.888915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.061580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.057912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.047659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.044900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.877708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.691255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.484389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.471649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.504264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.248881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.498630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.397972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.533903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.759876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.879384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.858886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.807861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.778700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.975595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.825619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.846752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.809546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.790432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.720454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.736400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.566375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.432498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.201645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.169985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.044383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.951526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.865171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.735641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.471945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.436679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.350038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.405706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.451139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.430682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.439991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.417528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.409774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.495599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.407620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.624251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.715540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.869487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.944521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.841089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.106336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.215062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.094643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.165457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.952891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.985458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.311636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.231686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.237313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.181421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.456269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.411415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.455593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.442605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.442107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.434510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.617093] LustreError: 172156:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6206.627286] LustreError: 172156:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6206.638431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.610642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.581529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.496251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.364659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.151929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.991507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.873379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.749663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.480582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.645660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.751417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.656739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.068227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.036782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.096830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.027781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.412765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.301563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.283766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.384158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.405459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.729472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.808379] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6229.821939] Lustre: Skipped 252 previous similar messages [ 6229.843095] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6229.861959] Lustre: Skipped 252 previous similar messages [ 6229.869139] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6229.875139] Lustre: Skipped 252 previous similar messages [ 6229.880806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.892869] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6229.906895] Lustre: Skipped 252 previous similar messages [ 6230.763596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.057468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.062399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.140344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.222206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.384272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.576648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.538339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.202882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.157103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.097290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.168483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.345113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.357580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.426821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.418041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.411487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.208301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.314951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.360346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.602164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.300118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.308951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.330087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.370836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.223598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.324081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.274709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.495838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.573504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.674274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.866631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.010805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.255091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.427730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.792184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.112121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.491546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.787446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.502565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.179203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.606546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.813766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.996829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.137922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.217832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.134142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.021772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.084842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.000262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.977306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.907866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.890984] LustreError: 172156:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6287.900498] LustreError: 172156:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6287.910542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.015218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.137068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.998275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.038552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.120526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.897597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.852945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.828397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.822603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.634463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.445518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.086438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.941266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.837030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.739073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.514643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.486330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.558717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.442851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.510594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.388426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.606701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.662208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.708366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.705350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.481436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.498806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.421594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.752717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.774521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.924868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.358465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.916516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.100256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.689362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.923604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.350970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.533892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.638221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.602611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.803271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.274336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.774487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.998400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.235256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.349703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.402154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.353220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.393237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.479630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.489390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.253977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.234633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.165478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.917483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.920733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.926926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.068239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.958345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.884581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.739941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.894887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.747080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.134731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.589198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.844751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.668891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.745860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.745410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.115789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.063223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.351745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.317147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.380135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.156240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.174520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.993498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.836292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.606995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.587226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.757463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.553098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.374515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.269614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.235341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.165871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.018586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.242530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.258130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.196553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.178549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.616682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.825646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.082129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.199485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.161202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.130810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.147346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.161957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.339418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.400510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.421293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.573974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.763963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.873693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.880789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.070615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.058183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.175567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.314795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.158319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.166550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.142482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.343461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.355333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.367484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.270211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.263925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.083879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.120663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.976542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.955796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.910716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.044917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.847496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.795488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.934511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.005326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.959990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.072563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.953791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.961090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.983512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.009305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.193429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.352032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.304607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.280988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.151563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.507738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.624055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.695144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.632261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.581289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.938527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.264030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.090928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.221655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.382912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.579030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.769047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.604542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.654650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.852853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.943842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.281637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.493448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.647498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.528323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.339049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.283421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.219993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.112674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.103340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.044446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.074695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.905622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.919172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.845300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.853089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.816314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.938125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.874834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.913480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.675507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.680569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.494483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.443206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.235874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.005203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.584191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.598600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.409175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.408461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.645925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.589617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.965406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.797356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.800495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.796992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.079289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.958259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.258970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.644848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.916478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.195576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.585899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.765991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.394924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.500890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.998657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.980563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.922757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.548815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.870833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.779684] LustreError: 172156:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6504.799819] LustreError: 172156:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6504.807462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.292581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.542537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.111475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.583260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.786634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.524275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.733709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.745937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.497806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.885509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.950925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.854662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.927974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.948321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.768291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.820149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.847549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.832215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.946983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.183354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.405791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.616697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.751041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.886925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.037597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.233552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.105815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.255079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.283243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.281080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.552021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.571415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.703584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.822843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.707078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.891416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.817400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.929327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.924945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.039394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.291176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.214455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.253238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.382103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.416239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.456509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.401863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.714291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.714470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.682157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.754121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.959712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.009974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.006986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.995530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.033313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.935692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.973344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.739066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.738802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.852651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.099905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.246358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.236970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.128922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.096711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.146593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.082263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.335328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.342669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.387709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.217533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.083470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.078453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.000787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.866878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.786211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.843080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.731500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.650737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.536864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.588241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.460703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.386992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.311864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.376894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.293158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.747133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.941523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.113940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.123898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.009438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.905295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.894860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.892194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.952521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.992808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.961441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.093216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.459782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.701530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.764401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.750594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.591099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.446674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.382840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.208051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.348158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.244088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.087039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.026741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.096150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.193328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.458484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.867610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.384747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.520370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.662057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.424542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.414885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.676846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.292739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.589629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.776729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.002953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.273945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.170845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.565380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.028303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.153376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.157244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.135899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.338393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.745349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.260203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.308340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.406931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.508898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.566343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.600399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.584730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.568711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.696122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.762355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.715986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.775979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.788077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.827605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.710618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.736465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.574984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.414808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.462937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.323357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.645824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.714391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.749739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.732955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.844316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.691714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.855353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.798389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.699810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.655345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.699150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.492949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.444461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.310423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.188205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.971043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.313322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.164744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.090766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.907747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.782582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.727140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.877407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.711079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.457080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.450712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.327081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.310871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.407702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.332147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.317583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.191767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.138893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.087734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.189975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.261809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.045598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.048648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.973895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.057741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.881931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.865677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.127617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.455725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.435373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.600513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.697766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.685853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.858283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.715201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.915703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.929833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.074278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.013643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.374421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.659573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.920433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.055713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.894900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.813890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.997310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.323186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.586535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.997604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.315474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.250268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.038529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.283404] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6742.287607] Lustre: Skipped 480 previous similar messages [ 6742.303459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6742.317356] Lustre: Skipped 480 previous similar messages [ 6742.323112] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6742.333388] Lustre: Skipped 480 previous similar messages [ 6742.341695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.360472] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6742.376386] Lustre: Skipped 480 previous similar messages [ 6743.624818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.749854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.251719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.775882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.881189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.011243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.047764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.111723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.413964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.494153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.709611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.833764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.598304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.417746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.601995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.725178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.768650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.889203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.099247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.212813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.141110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.209563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.990127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.007353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.865888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.963895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.284137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.283978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.166655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.903682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.758783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.583690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.761785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.479704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.520508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.255097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.084743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.787638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.674498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.672950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.612399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.512730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.457856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.394830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.387848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.276934] LustreError: 173958:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7211.282798] LustreError: 172156:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7892.462593] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7892.469282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7892.481736] Lustre: Skipped 45 previous similar messages [ 7892.490921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7893.473633] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7893.496390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7895.521439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7895.538325] Lustre: Skipped 1 previous similar message [ 7900.649728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7900.675665] Lustre: Skipped 1 previous similar message [ 7905.249814] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7905.550466] Lustre: server umount lustre-OST0000 complete [ 7909.856762] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7909.868618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7916.001627] LustreError: 172136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7916.010197] LustreError: 172136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7916.130166] Lustre: server umount lustre-MDT0000 complete [ 7920.217430] LustreError: 216196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788430478 with bad export cookie 4816879966699209373 [ 7920.227603] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7920.235527] LustreError: 216196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7920.607472] Lustre: server umount lustre-MDT0001 complete [ 7934.464395] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 7937.813489] Key type lgssc unregistered [ 7938.120206] LNet: 227438:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7938.129174] LNetError: 227438:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7938.164475] LNet: Removed LNI 192.168.202.101@tcp [ 7939.142641] Key type .llcrypt unregistered [ 7939.146244] Key type ._llcrypt unregistered [ 7968.897903] Key type ._llcrypt registered [ 7968.901288] Key type .llcrypt registered [ 7969.040307] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 7970.700128] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7970.745152] alg: No test for adler32 (adler32-zlib) [ 7972.044644] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 7972.444332] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 7974.143808] Key type lgssc registered [ 7975.426136] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7984.527969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7984.553405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7986.089281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7990.454581] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 7995.365065] LustreError: 228850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7995.395759] LustreError: 228850:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8000.489307] LustreError: 228851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8000.758663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8001.181265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8006.729429] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8016.505457] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8024.329807] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8033.923127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8034.378497] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8042.096675] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8051.190202] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8055.986083] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8061.952689] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8064.842148] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8064.861683] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8064.876988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8064.894098] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8064.897811] LustreError: 228870:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8068.016500] Lustre: lustre-OST0000: Not available for connect from 192.168.202.1@tcp (stopping) [ 8071.652431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8071.658231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8071.666672] Lustre: Skipped 1 previous similar message [ 8071.681551] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8073.107046] Lustre: lustre-OST0000: Not available for connect from 192.168.202.1@tcp (stopping) [ 8073.122371] Lustre: Skipped 1 previous similar message [ 8076.769681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8076.783073] Lustre: Skipped 1 previous similar message [ 8081.891404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8081.921960] Lustre: Skipped 1 previous similar message [ 8082.401117] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8082.538689] Lustre: server umount lustre-OST0000 complete [ 8084.967099] LustreError: 228870:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8119.578101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8119.864363] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8127.512409] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8136.237886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8136.278111] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8136.279560] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8137.418205] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8141.150477] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8152.146283] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff975944669000.ost_server_uuid 50 [ 8153.933360] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff975944669000.ost_server_uuid in FULL state after 0 sec [ 8159.200225] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8159.211792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8159.254116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8159.266242] Lustre: Skipped 1 previous similar message [ 8161.789298] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8164.425420] Lustre: server umount lustre-OST0000 complete [ 8169.952633] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8169.957694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8174.707509] Lustre: server umount lustre-MDT0000 complete [ 8177.128947] LustreError: 228846:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8177.153584] LustreError: 228846:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8179.184876] LustreError: 228832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788430737 with bad export cookie 13496947912847320678 [ 8179.189627] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8179.201888] LustreError: 228832:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8179.915675] Lustre: server umount lustre-MDT0001 complete [ 8193.371565] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 8196.684603] Key type lgssc unregistered [ 8197.178787] LNet: 234064:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8197.202925] LNetError: 234064:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8197.229006] LNet: Removed LNI 192.168.202.101@tcp [ 8198.165301] Key type .llcrypt unregistered [ 8198.167535] Key type ._llcrypt unregistered [ 8235.825464] Key type ._llcrypt registered [ 8235.828425] Key type .llcrypt registered [ 8235.925875] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 8237.170492] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8237.229569] alg: No test for adler32 (adler32-zlib) [ 8238.851663] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 8239.232335] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 8240.975147] Key type lgssc registered [ 8243.581494] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8258.432187] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8258.457586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8259.895282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8263.686949] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8269.282837] LustreError: 235796:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8269.300067] LustreError: 235796:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8273.704632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8274.045922] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8279.665910] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8283.233455] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8291.771373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8292.140111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8298.582378] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8304.462791] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8308.158931] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8308.273868] Lustre: lustre-OST0001: new disk, initializing [ 8308.278094] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8308.283816] Lustre: 238106:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8308.395639] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8315.666335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8315.677855] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8315.819064] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8316.087732] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8328.106465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8335.871708] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8355.809655] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8355.817127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8355.837835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8356.835706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8356.836441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8356.849449] Lustre: Skipped 2 previous similar messages [ 8356.864908] Lustre: Skipped 3 previous similar messages [ 8359.936389] Lustre: server umount lustre-MDT0000 complete [ 8361.964316] LustreError: 235793:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8361.994056] LustreError: 235793:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8363.603043] LustreError: 235776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788430921 with bad export cookie 1966391859573667330 [ 8363.603570] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8363.608522] LustreError: 235776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8363.848583] Lustre: server umount lustre-MDT0001 complete [ 8378.729669] Lustre: server umount lustre-OST0000 complete [ 8393.558130] Lustre: server umount lustre-OST0001 complete [ 8418.215445] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 8429.180594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8429.752363] LustreError: 241214:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8429.833124] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8435.078448] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8435.168854] LustreError: 241215:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8439.283029] LustreError: 241214:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8443.772691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8444.325904] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8449.114266] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8452.716584] Lustre: 242357:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8462.035806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8462.699762] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8469.868145] LustreError: 242710:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8469.914472] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8471.433975] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8475.065747] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8487.165960] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8493.088363] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8496.051302] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8505.705017] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8512.275149] Lustre: 244229:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8526.307714] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8526.326417] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8526.343972] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8528.864935] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8528.869360] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8528.875424] Lustre: Skipped 2 previous similar messages [ 8528.883824] Lustre: Skipped 2 previous similar messages [ 8532.165338] Lustre: server umount mylustre-MDT0000 complete [ 8533.987956] LustreError: 241948:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8534.016947] LustreError: 241948:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8536.821359] LustreError: 241190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788431094 with bad export cookie 1966391859573670487 [ 8536.823748] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8536.847795] LustreError: 241190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8537.411842] Lustre: server umount mylustre-MDT0001 complete [ 8552.849619] Lustre: server umount mylustre-OST0000 complete [ 8554.339806] Lustre: 235057:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788431096/real 1788431096] req@ffff8bd58565aa00 x1875305611588224/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788431112 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8554.388563] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8556.512664] Lustre: 235055:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788431098/real 1788431098] req@ffff8bd48fbcbb80 x1875305611588480/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788431114 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8558.612687] Lustre: server umount mylustre-OST0001 complete [ 8586.790516] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 8600.261250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8600.705042] LustreError: 246539:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8600.806667] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8600.810446] Lustre: Skipped 1 previous similar message [ 8606.264780] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8615.638556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8621.394692] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8624.820363] Lustre: 247682:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8632.252474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8639.712812] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8640.888393] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8646.021340] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8648.414436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8653.817611] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 8656.210449] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8665.427602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8669.974496] Lustre: 249557:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8679.398262] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8679.407251] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8679.409391] Lustre: Skipped 3 previous similar messages [ 8679.414275] Lustre: Skipped 3 previous similar messages [ 8684.516536] LustreError: 246540:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8684.547799] LustreError: 246540:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 8684.715058] Lustre: server umount tfs-MDT0000 complete [ 8689.178678] LustreError: 246516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788431247 with bad export cookie 1966391859573672811 [ 8689.184654] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8689.197290] LustreError: 246516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8689.684735] Lustre: server umount tfs-MDT0001 complete [ 8705.503354] Lustre: 235057:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788431247/real 1788431247] req@ffff8bd49c854e00 x1875305611645440/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788431263 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8705.584360] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8705.897118] Lustre: server umount tfs-OST0000 complete [ 8710.633724] Lustre: server umount tfs-OST0001 complete [ 8738.624706] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 8750.905657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8751.427098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8751.429989] Lustre: Skipped 3 previous similar messages [ 8755.880260] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8765.508227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8770.971562] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8774.166634] Lustre: 252949:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8782.701848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8791.218533] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8791.555379] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8797.774370] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8802.051910] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8807.424804] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8808.379467] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8817.689972] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8822.076284] Lustre: 254820:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8831.410910] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:29:46 (1788431386) [ 8852.963494] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8852.984358] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8853.017123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8863.199265] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8863.374292] Lustre: server umount lustre-MDT0000 complete [ 8863.717270] LustreError: 251803:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8863.746950] LustreError: 251803:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 8867.287980] LustreError: 254132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788431425 with bad export cookie 1966391859573674848 [ 8867.297660] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8867.306622] LustreError: 254132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8872.864535] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8872.870667] Lustre: Skipped 11 previous similar messages [ 8873.891687] Lustre: server umount lustre-MDT0001 complete [ 8885.132114] Lustre: server umount lustre-OST0000 complete [ 8896.357080] Lustre: server umount lustre-OST0001 complete [ 8904.358455] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 8916.494933] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 8960.251207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8960.518614] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8960.549537] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8960.641948] Lustre: lustre-MDT0000: new disk, initializing [ 8960.696908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8960.704655] Lustre: Skipped 3 previous similar messages [ 8960.720838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8965.242393] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8976.630118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8976.739337] Lustre: 258011:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8976.797394] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8976.801394] Lustre: Skipped 1 previous similar message [ 8976.864574] Lustre: lustre-MDT0001: new disk, initializing [ 8976.980407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8976.992645] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8982.367891] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 8995.014470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8995.460526] Lustre: lustre-OST0000: new disk, initializing [ 8995.472860] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8995.485164] Lustre: 259593:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8997.142960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8997.154853] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8997.376637] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9003.739992] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9016.855859] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9017.089601] Lustre: lustre-OST0001: new disk, initializing [ 9017.093197] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9017.108508] Lustre: 260465:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9018.455508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9018.468440] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9018.565493] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9024.494729] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9038.567293] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:33:14 (1788431594) [ 9057.619400] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:33:33 (1788431613) [ 9074.146561] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9074.167243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9074.194631] Lustre: Skipped 5 previous similar messages [ 9074.224044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9079.621499] Lustre: server umount lustre-OST0000 complete [ 9084.384824] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9084.402376] LustreError: Skipped 1 previous similar message [ 9090.094515] Lustre: server umount lustre-MDT0000 complete [ 9093.650926] LustreError: 260464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788431651 with bad export cookie 1966391859573676731 [ 9093.659300] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9093.664035] LustreError: 260464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9100.103375] Lustre: server umount lustre-MDT0001 complete [ 9112.522147] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 9187.753204] Lustre: server umount lustre-OST0001 complete [ 9195.678880] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 9213.707917] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 9254.099833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9254.397810] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9254.427983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9254.520763] Lustre: lustre-MDT0000: new disk, initializing [ 9254.631568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9254.639256] Lustre: Skipped 3 previous similar messages [ 9254.665806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9259.936124] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9270.670405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9270.877163] Lustre: 265322:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9270.966500] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9270.977542] Lustre: Skipped 1 previous similar message [ 9271.099273] Lustre: lustre-MDT0001: new disk, initializing [ 9271.256921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9271.271069] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9276.246340] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9286.903779] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9293.542929] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9302.527498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9302.867793] Lustre: lustre-OST0000: new disk, initializing [ 9302.875777] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9302.884558] Lustre: 267291:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9304.241566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9304.291220] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9304.445760] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9310.455543] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9320.256337] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9330.145103] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9330.159347] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9330.174626] Lustre: Skipped 6 previous similar messages [ 9330.180082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9330.183831] Lustre: Skipped 9 previous similar messages [ 9334.529400] Lustre: server umount lustre-OST0000 complete [ 9352.159238] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9354.402324] Lustre: server umount lustre-MDT0000 complete [ 9355.761716] LustreError: 265329:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9355.783099] LustreError: 265329:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 9358.485952] LustreError: 265312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788431916 with bad export cookie 1966391859573678341 [ 9358.493512] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9358.499808] LustreError: 265312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9358.793747] Lustre: server umount lustre-MDT0001 complete [ 9370.327711] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 9372.762648] Key type lgssc unregistered [ 9373.010977] LNet: 269052:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9373.015640] LNetError: 269052:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9373.033282] LNet: Removed LNI 192.168.202.101@tcp [ 9373.766168] Key type .llcrypt unregistered [ 9373.771304] Key type ._llcrypt unregistered [ 9405.684787] Key type ._llcrypt registered [ 9405.688263] Key type .llcrypt registered [ 9405.848277] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 9407.021862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9407.193695] alg: No test for adler32 (adler32-zlib) [ 9408.238927] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 9408.509718] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 9410.240768] Key type lgssc registered [ 9411.523609] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9419.214421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9419.232172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9420.829212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9425.292529] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9431.012315] LustreError: 270515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9431.033617] LustreError: 270515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9434.946347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9435.308062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9439.994677] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9447.470205] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9454.094849] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9461.040796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9461.530274] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9465.670553] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9468.590819] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9476.233920] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9477.305952] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9481.185261] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9481.222117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9485.586572] Lustre: server umount lustre-OST0000 complete [ 9491.426161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9491.439894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9491.448988] Lustre: Skipped 1 previous similar message [ 9491.481278] Lustre: Skipped 2 previous similar messages [ 9495.687630] Lustre: server umount lustre-MDT0000 complete [ 9496.546920] LustreError: 270515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9496.566805] LustreError: 270515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9499.371932] LustreError: 270497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788432057 with bad export cookie 358095291855148268 [ 9499.377029] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9499.378534] LustreError: 270497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9499.716416] Lustre: server umount lustre-MDT0001 complete [ 9510.831472] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 9513.505320] Key type lgssc unregistered [ 9513.860771] LNet: 274099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9513.870623] LNetError: 274099:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9513.899948] LNet: Removed LNI 192.168.202.101@tcp [ 9514.779353] Key type .llcrypt unregistered [ 9514.782403] Key type ._llcrypt unregistered [ 9532.802283] Key type ._llcrypt registered [ 9532.804552] Key type .llcrypt registered [ 9532.902325] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 9533.773464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9533.867300] alg: No test for adler32 (adler32-zlib) [ 9534.943465] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 9535.172329] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 9536.895162] Key type lgssc registered [ 9538.030232] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9547.374483] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9547.383460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9548.452808] Lustre: 275301:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9563.936837] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9569.055588] LustreError: 275289:0:(mgc_request.c:317:do_config_log_add()) mgc192.168.202.101@tcp: failed processing log, type 4: rc = -110 [ 9573.862047] Lustre: 275303:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9579.359417] Lustre: 275289:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -110 [ 9589.687812] Lustre: 275301:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9590.751451] Lustre: 275305:0:(mgc_request.c:2378:mgc_process_log()) mgc192.168.202.101@tcp: IR log lustre-mdtir failed, not fatal: rc = -110 [ 9594.848247] Lustre: 275303:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9599.969836] Lustre: 275301:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9599.970333] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9600.993489] LustreError: 275305:0:(mgc_request.c:718:do_requeue()) failed processing log: -110 [ 9607.398711] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9610.209707] Lustre: 275301:0:(ldlm_lib.c:1251:target_handle_connect()) MGS: refusing connection from 0@lo: this is an MGS follower, which only serves clients that support it: rc = -19 [ 9610.237181] Lustre: 275301:0:(ldlm_lib.c:1251:target_handle_connect()) Skipped 1 previous similar message [ 9615.897784] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9621.473302] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9621.666818] Lustre: server umount lustre-OST0000 complete [ 9642.382928] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [ 9645.884461] Key type lgssc unregistered [ 9646.171833] LNet: 276929:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9646.180619] LNetError: 276929:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9646.199725] LNet: Removed LNI 192.168.202.101@tcp [ 9647.166960] Key type .llcrypt unregistered [ 9647.171284] Key type ._llcrypt unregistered [ 9688.782604] Key type ._llcrypt registered [ 9688.786707] Key type .llcrypt registered [ 9688.878582] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 9689.551249] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9689.573565] alg: No test for adler32 (adler32-zlib) [ 9690.604596] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [ 9690.798736] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [ 9692.455169] Key type lgssc registered [ 9693.477793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9705.903340] loop0: detected capacity change from 0 to 204800000 [ 9706.142179] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9706.152956] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9706.574722] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9707.767921] loop0: detected capacity change from 0 to 204800000 [ 9707.877286] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9707.883562] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9708.165798] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9709.053312] loop0: detected capacity change from 0 to 204800000 [ 9709.267384] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9709.285915] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9709.421714] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9710.583753] loop0: detected capacity change from 0 to 204800000 [ 9710.726657] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9711.737747] loop0: detected capacity change from 0 to 204800000 [ 9711.762511] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9715.103920] loop0: detected capacity change from 0 to 204800000 [ 9715.133738] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9718.424115] loop0: detected capacity change from 0 to 204800000 [ 9718.450677] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9722.145417] loop0: detected capacity change from 0 to 204800000 [ 9722.169214] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9726.498116] loop0: detected capacity change from 0 to 204800000 [ 9726.645685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9726.709444] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9726.827840] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9726.846300] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9726.862536] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9726.873818] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9726.885967] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9726.896842] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9726.908047] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9726.924746] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9726.934317] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9726.943825] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9726.953794] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9726.960474] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9726.965974] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9726.974654] Lustre: 280341:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9733.218990] Lustre: server umount MGS complete [ 9734.258625] loop0: detected capacity change from 0 to 204800000 [ 9734.438773] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9735.255600] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9735.317856] Lustre: lustre-MDT0000: Aborting client recovery [ 9736.188971] loop1: detected capacity change from 0 to 204800000 [ 9736.349227] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9736.513915] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9736.529698] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9736.551405] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9736.561667] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9736.578899] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9736.595996] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9736.606175] Lustre: 280773:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9736.618645] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9736.819630] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9736.902492] Lustre: lustre-MDT0001: Aborting client recovery [ 9738.035119] loop2: detected capacity change from 0 to 204800000 [ 9738.223284] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9738.324358] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9738.338351] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9738.353693] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9738.373765] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9738.391765] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9738.411025] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9738.429124] Lustre: 280852:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9738.528747] Lustre: Found index 0 for lustre-OST0000, updating log [ 9738.567812] Lustre: 280869:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9738.621297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9738.649642] Lustre: lustre-OST0000: Aborting recovery [ 9739.695617] loop3: detected capacity change from 0 to 204800000 [ 9739.808491] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9739.887093] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9739.898172] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9739.925698] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9739.947502] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9739.971775] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9739.981978] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9739.989459] Lustre: 280934:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9739.998573] Lustre: Found index 1 for lustre-OST0001, updating log [ 9740.029376] Lustre: 280939:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9740.103327] Lustre: lustre-OST0001: Aborting recovery [ 9745.188693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9745.194865] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9745.239301] LustreError: 280717:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9745.249323] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9745.298979] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9746.224219] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9746.230417] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9748.129424] LustreError: 280684:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 642us: rc = -2 [ 9748.655483] LustreError: 280685:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 2155us: rc = -2 [ 9748.690431] LustreError: 280685:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 14 previous similar messages [ 9762.272521] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9762.277645] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9762.291058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9766.879897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9766.880052] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9766.905541] Lustre: Skipped 2 previous similar messages [ 9768.298665] Lustre: server umount lustre-MDT0000 complete [ 9769.336352] LustreError: 280668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788432327 with bad export cookie 1452743216382935877 [ 9769.342854] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9769.354780] LustreError: 280668:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9771.999779] LustreError: 280683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9772.000291] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9772.007189] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9772.007197] Lustre: Skipped 3 previous similar messages [ 9772.019073] LustreError: 280683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9772.031178] Lustre: Skipped 1 previous similar message [ 9775.008828] LustreError: 280682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9775.015164] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9775.044103] LustreError: 280682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9775.087778] Lustre: Skipped 2 previous similar messages [ 9776.073428] Lustre: server umount lustre-MDT0001 complete [ 9783.974444] Lustre: server umount lustre-OST0000 complete [ 9791.345278] Lustre: server umount lustre-OST0001 complete [ 9802.973865] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:45:58 (1788432358) [ 9836.732862] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [ 9849.653334] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [ 9891.354599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9891.626056] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9891.684220] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9891.791683] Lustre: lustre-MDT0000: new disk, initializing [ 9891.916791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9891.920898] Lustre: Skipped 1 previous similar message [ 9891.935731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9896.502427] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9907.472796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9907.600766] Lustre: 285007:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9907.738793] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9907.757746] Lustre: Skipped 1 previous similar message [ 9907.969852] Lustre: lustre-MDT0001: new disk, initializing [ 9908.137197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9908.183532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9908.203787] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9912.416867] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9922.364590] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9929.641822] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9937.370141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9937.609052] Lustre: lustre-OST0000: new disk, initializing [ 9937.616320] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9937.633633] Lustre: 286979:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9937.756443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9943.896672] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [ 9947.814796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9947.822947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9947.953124] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9954.703158] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9961.094882] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9973.486163] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9983.550821] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9989.774258] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10030.461976] Lustre: server umount lustre-OST0000 complete [10035.174384] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10035.190652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10043.521259] Lustre: server umount lustre-MDT0000 complete [10045.413668] LustreError: 285017:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10045.433717] LustreError: 285017:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10047.544203] LustreError: 288741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788432605 with bad export cookie 1452743216382938670 [10047.545952] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10047.564200] LustreError: 288741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10057.356403] Lustre: server umount lustre-MDT0001 complete [10065.977524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10070.373216] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10075.260364] Lustre: server umount MGS complete [10086.289930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10086.799320] LustreError: 290949:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10086.824854] LustreError: 290962:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10086.917234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10091.479682] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10092.001364] LustreError: 290950:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10100.066447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10105.237505] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10113.702291] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10119.746930] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10129.353716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10129.849829] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10129.855449] Lustre: Skipped 1 previous similar message [10136.720471] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10143.741337] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10175.976755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10175.983079] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10175.998507] Lustre: Skipped 1 previous similar message [10176.000599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10176.025645] Lustre: Skipped 4 previous similar messages [10179.340880] Lustre: server umount lustre-OST0000 complete [10186.217071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10186.225974] Lustre: Skipped 2 previous similar messages [10189.588898] Lustre: server umount lustre-MDT0000 complete [10191.328952] LustreError: 291678:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10191.357126] LustreError: 291678:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10192.734952] LustreError: 293676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788432750 with bad export cookie 1452743216382940875 [10192.740694] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10192.745835] LustreError: 293676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10192.951564] Lustre: server umount lustre-MDT0001 complete [10205.796667] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [10208.740530] Key type lgssc unregistered [10208.996678] LNet: 294892:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10209.000465] LNetError: 294892:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10209.017215] LNet: Removed LNI 192.168.202.101@tcp [10209.757101] Key type .llcrypt unregistered [10209.759623] Key type ._llcrypt unregistered [10249.656467] Key type ._llcrypt registered [10249.658766] Key type .llcrypt registered [10249.772319] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [10267.942249] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [10268.954213] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10269.084448] alg: No test for adler32 (adler32-zlib) [10270.240868] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [10270.585297] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [10272.303179] Key type lgssc registered [10273.364768] Lustre: Echo OBD driver; http://www.lustre.org/ [10307.310450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10307.336366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10308.656711] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10308.688364] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10308.788302] Lustre: lustre-MDT0000: new disk, initializing [10308.890255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10308.916794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10313.752448] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10324.028124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10324.086296] Lustre: 298848:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10324.160310] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10324.165359] Lustre: Skipped 1 previous similar message [10324.236777] Lustre: lustre-MDT0001: new disk, initializing [10324.310692] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10324.336714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10324.349059] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10328.803323] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10339.205963] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10344.965078] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10352.488856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10352.758384] Lustre: lustre-OST0000: new disk, initializing [10352.761686] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10352.768781] Lustre: 300814:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10352.859067] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10359.021732] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10360.993851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10361.004115] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10361.131378] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10369.605766] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10376.452897] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10389.422591] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10397.770942] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10448.365465] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10448.371770] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10448.371941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10448.403464] Lustre: Skipped 1 previous similar message [10450.762445] Lustre: server umount lustre-OST0000 complete [10457.581890] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10457.599846] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10457.631762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10457.641113] Lustre: Skipped 1 previous similar message [10462.713362] Lustre: server umount lustre-MDT0000 complete [10466.364968] LustreError: 298840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788433024 with bad export cookie 18248378059727950425 [10466.368888] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10466.371607] LustreError: 298840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10466.719907] Lustre: server umount lustre-MDT0001 complete [10475.810213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10480.736593] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10485.849772] Lustre: server umount MGS complete [10497.384994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10497.765434] LustreError: 304733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10497.783943] LustreError: 304733:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10497.794190] LustreError: 304746:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10497.851698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10503.149941] LustreError: 304734:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10503.188371] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10507.243537] LustreError: 304734:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10512.353877] LustreError: 304733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10513.597786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10514.304356] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10519.426719] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10529.737398] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10539.061026] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10548.108133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10548.383758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10556.781832] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10565.635253] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10604.006101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10604.019754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10604.023434] Lustre: Skipped 2 previous similar messages [10604.023938] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10604.043771] Lustre: Skipped 3 previous similar messages [10607.461848] Lustre: server umount lustre-OST0000 complete [10611.697164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10611.713290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10611.734607] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10617.179459] Lustre: server umount lustre-MDT0000 complete [10619.371984] LustreError: 304734:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10619.408601] LustreError: 304734:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10621.449761] LustreError: 307463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788433179 with bad export cookie 18248378059727952497 [10621.459549] LustreError: 307463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10621.460510] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10621.863572] Lustre: server umount lustre-MDT0001 complete [10634.064795] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [10636.509678] Key type lgssc unregistered [10636.824314] LNet: 308680:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10636.839741] LNetError: 308680:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10636.868293] LNet: Removed LNI 192.168.202.101@tcp [10638.131362] Key type .llcrypt unregistered [10638.133539] Key type ._llcrypt unregistered [10661.129607] Key type ._llcrypt registered [10661.135914] Key type .llcrypt registered [10661.320894] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [10677.902563] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [10679.404952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10679.422094] alg: No test for adler32 (adler32-zlib) [10680.380979] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [10680.629778] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [10682.303208] Key type lgssc registered [10683.308444] Lustre: Echo OBD driver; http://www.lustre.org/ [10710.804734] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [10728.515255] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10728.538754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10729.879266] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10729.929979] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10730.018886] Lustre: lustre-MDT0000: new disk, initializing [10730.097937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10730.119586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10734.718585] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10746.819541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10747.235848] Lustre: lustre-OST0000: new disk, initializing [10747.240329] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10747.252512] Lustre: Skipped 1 previous similar message [10747.265310] Lustre: 312744:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10747.447042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10754.081072] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [10754.156393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10754.170393] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10754.216447] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11675.120483] Lustre: server umount lustre-MDT0000 complete [11680.121143] LustreError: 311932:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788434237 with bad export cookie 11631744376208377501 [11680.134722] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11689.779572] Lustre: server umount lustre-OST0000 complete [11722.852182] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [11736.065385] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [11782.532171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11782.741455] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11782.769198] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11782.845341] Lustre: lustre-MDT0000: new disk, initializing [11782.904931] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11782.915465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11787.085334] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11800.158805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11800.274495] Lustre: 317094:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11800.460841] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11800.463584] Lustre: Skipped 1 previous similar message [11800.625864] Lustre: lustre-MDT0001: new disk, initializing [11800.869365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11800.932392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11800.959454] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11807.903041] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11820.046342] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11826.924040] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11836.442705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11837.018556] Lustre: lustre-OST0000: new disk, initializing [11837.026951] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11837.044661] Lustre: 319063:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11837.286473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11842.763183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11842.783750] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11842.987682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11845.715374] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11856.498287] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11863.180890] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11863.559219] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11868.493590] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11868.688201] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11873.776380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11873.780814] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11873.790073] LustreError: Skipped 1 previous similar message [11873.790819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11873.830752] Lustre: Skipped 1 previous similar message [11876.847864] Lustre: server umount lustre-OST0000 complete [11882.977717] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11882.996707] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11883.021683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11883.035985] Lustre: Skipped 1 previous similar message [11884.003243] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11889.122110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11889.128370] Lustre: Skipped 3 previous similar messages [11893.217251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11893.223150] Lustre: Skipped 1 previous similar message [11895.265911] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11895.550788] Lustre: server umount lustre-MDT0000 complete [11899.258085] LustreError: 317085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788434457 with bad export cookie 11631744376211318355 [11899.281021] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11899.293654] LustreError: 317085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11899.592684] Lustre: server umount lustre-MDT0001 complete [11903.232828] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11904.985518] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:21:00 (1788434460) [11911.255825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11911.722157] LustreError: 321377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11911.735878] LustreError: 321377:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11911.794931] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11916.807092] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11917.286044] LustreError: 321378:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11921.380957] LustreError: 321377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11926.168555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11926.601224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11931.609963] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11939.874346] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11946.360075] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11954.972851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11955.346516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11962.480918] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11970.797914] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11978.590453] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11978.799776] Lustre: lustre-OST0001: new disk, initializing [11978.803739] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11978.808799] Lustre: 324216:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11978.908731] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11985.041843] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [11985.615603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11985.632419] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11985.684750] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11986.732211] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11986.756660] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11987.822383] LustreError: 324608:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11987.833895] LustreError: 324608:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11989.861638] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11989.898462] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11994.024133] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11994.041351] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11995.869402] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11996.099205] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12002.278893] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12002.294524] LustreError: 324599:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12002.896675] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff975945d4b800.ost_server_uuid 50 [12004.812358] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff975945d4b800.ost_server_uuid in FULL state after 0 sec [12011.092581] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff975945d4b800.ost_server_uuid 50 [12012.995747] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff975945d4b800.ost_server_uuid in IDLE state after 0 sec [12020.215371] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12029.923768] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12029.937110] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12029.964745] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12035.235158] Lustre: server umount lustre-OST0001 complete [12036.504885] LustreError: 323197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.1@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12042.220948] LustreError: 323199:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12042.260254] LustreError: 323199:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12043.751713] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12043.752808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12043.766413] LustreError: Skipped 2 previous similar messages [12043.799192] Lustre: Skipped 1 previous similar message [12043.803262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12043.817860] Lustre: Skipped 4 previous similar messages [12049.905736] Lustre: server umount lustre-OST0000 complete [12057.569240] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12057.590921] Lustre: Skipped 1 previous similar message [12062.689109] LustreError: 321377:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12062.731540] LustreError: 321377:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12063.778985] Lustre: server umount lustre-MDT0000 complete [12068.394826] LustreError: 324215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788434626 with bad export cookie 11631744376211319279 [12068.410541] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12068.416397] LustreError: 324215:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12068.931987] Lustre: server umount lustre-MDT0001 complete [12083.300402] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [12086.599397] Key type lgssc unregistered [12087.054635] LNet: 326909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12087.067920] LNetError: 326909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12087.095067] LNet: Removed LNI 192.168.202.101@tcp [12088.190982] Key type .llcrypt unregistered [12088.193397] Key type ._llcrypt unregistered [12119.865948] Key type ._llcrypt registered [12119.870960] Key type .llcrypt registered [12120.065359] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [12121.396594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12121.819474] alg: No test for adler32 (adler32-zlib) [12123.341897] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [12123.662556] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [12125.415217] Key type lgssc registered [12127.842298] Lustre: Echo OBD driver; http://www.lustre.org/ [12139.549369] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12139.563951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12141.366729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12146.734953] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12151.783328] LustreError: 328329:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12151.805195] LustreError: 328329:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12155.872773] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12156.897926] LustreError: 328330:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12162.024129] LustreError: 328329:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12163.933253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12164.230847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12169.418118] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12180.475850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12181.249510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12186.345182] LustreError: 330011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12186.351609] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12190.176936] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12193.783476] LustreError: 330197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12193.846107] LustreError: 330197:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12201.647651] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12204.007836] LustreError: 330012:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12204.033306] LustreError: 330012:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12208.706225] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12208.987169] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12214.274987] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12215.767225] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12225.221634] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12235.194637] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12235.531312] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12347.367969] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12347.376440] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12347.393629] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12350.796765] Lustre: server umount lustre-OST0001 complete [12352.491220] LustreError: 330197:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12352.528493] LustreError: 330197:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12360.673094] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12360.679445] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12360.693237] Lustre: Skipped 1 previous similar message [12360.704374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12360.714263] Lustre: Skipped 1 previous similar message [12362.722682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12362.729732] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12362.757462] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12366.273268] Lustre: server umount lustre-OST0000 complete [12372.968937] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12372.986816] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12372.988419] Lustre: Skipped 1 previous similar message [12373.004850] Lustre: Skipped 2 previous similar messages [12378.093785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12378.101771] Lustre: Skipped 1 previous similar message [12385.247351] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12385.453733] Lustre: server umount lustre-MDT0000 complete [12387.301662] LustreError: 329253:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12387.339613] LustreError: 329253:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12389.704172] LustreError: 328310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788434947 with bad export cookie 12269671999042812168 [12389.707827] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12389.715972] LustreError: 328310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12390.300184] Lustre: server umount lustre-MDT0001 complete [12404.339788] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [12407.723910] Key type lgssc unregistered [12408.169170] LNet: 333279:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12408.183533] LNetError: 333279:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12408.203859] LNet: Removed LNI 192.168.202.101@tcp [12409.947899] Key type .llcrypt unregistered [12409.952519] Key type ._llcrypt unregistered [12466.505402] Key type ._llcrypt registered [12466.512265] Key type .llcrypt registered [12466.670437] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [12468.461782] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12468.551333] alg: No test for adler32 (adler32-zlib) [12469.724946] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [12470.016962] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [12471.703178] Key type lgssc registered [12473.254465] Lustre: Echo OBD driver; http://www.lustre.org/ [12484.971736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12484.991613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12486.485231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12491.035868] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12496.881597] LustreError: 335569:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12496.916804] LustreError: 335569:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12501.006447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12501.438866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12506.185806] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12509.444377] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12518.701986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12519.321548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12526.536731] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12529.390126] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12531.658230] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12531.714100] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12531.724453] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12536.803624] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12536.843677] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12541.931971] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12541.943223] LustreError: 337058:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12542.018388] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12542.299255] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12547.621380] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12553.469717] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12566.580300] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12577.762428] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12598.752937] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12598.766135] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12598.772975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12598.776777] Lustre: Skipped 3 previous similar messages [12598.788214] Lustre: Skipped 3 previous similar messages [12601.951681] Lustre: server umount lustre-MDT0000 complete [12603.878719] LustreError: 335569:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12603.912736] LustreError: 335569:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12606.148156] LustreError: 335550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788435163 with bad export cookie 17324388682190123308 [12606.162571] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12606.172184] LustreError: 335550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12606.566128] Lustre: server umount lustre-MDT0001 complete [12621.930347] Lustre: server umount lustre-OST0000 complete [12624.351166] Lustre: 334829:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788435166/real 1788435166] req@ffff8bd5b72d6d80 x1875310046949760/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788435182 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12624.390985] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12626.470690] Lustre: server umount lustre-OST0001 complete [12645.614985] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [12655.708591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12656.201258] LustreError: 340838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12656.312844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12661.232164] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12672.459833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12672.935400] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12677.451353] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12681.316266] Lustre: 341979:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12690.554270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12690.841040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12698.031829] LustreError: 342333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12698.076497] LustreError: 342333:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12698.768728] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12702.182167] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12711.785787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12717.576927] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12720.886201] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [12731.578593] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12735.506548] Lustre: 343853:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12743.143763] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12743.182094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12743.198351] Lustre: Skipped 3 previous similar messages [12746.290086] Lustre: server umount lustre-MDT0000 complete [12749.742654] LustreError: 340820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788435307 with bad export cookie 17324388682190125163 [12749.744333] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12749.760777] LustreError: 340820:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12750.302580] Lustre: server umount lustre-MDT0001 complete [12763.520801] Lustre: server umount lustre-OST0000 complete [12778.073934] Lustre: server umount lustre-OST0001 complete [12781.979827] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12783.960138] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12785.724077] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:35:41 (1788435341) [12813.083821] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [12823.565612] loop0: detected capacity change from 0 to 1099511627776 [12824.639432] SGI XFS with ACLs, security attributes, quota, no debug enabled [12824.645477] *** VALIDATE xfs *** [12824.672980] XFS (loop0): Mounting V5 Filesystem [12824.725423] XFS (loop0): Ending clean mount [12826.533776] loop1: detected capacity change from 0 to 18691697672192 [12955.685618] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [12966.869576] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [13005.122103] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13005.382288] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13005.410443] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13005.498853] Lustre: lustre-MDT0000: new disk, initializing [13005.586104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13005.598853] Lustre: Skipped 1 previous similar message [13005.610322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13010.013873] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13021.214315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13021.328197] Lustre: 348995:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13021.419209] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13021.426968] Lustre: Skipped 1 previous similar message [13021.549445] Lustre: lustre-MDT0001: new disk, initializing [13021.708260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13021.714982] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13026.759047] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13037.795761] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13044.238863] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13051.823846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13052.157858] Lustre: lustre-OST0000: new disk, initializing [13052.161603] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13052.167631] Lustre: 350964:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13053.538308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13053.556735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13053.714178] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13059.090621] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13068.818703] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13073.449311] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13073.756101] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13078.574970] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13078.724630] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13084.642497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13084.644981] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13084.649062] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13084.650564] LustreError: Skipped 1 previous similar message [13084.678511] Lustre: Skipped 4 previous similar messages [13086.849955] Lustre: server umount lustre-OST0000 complete [13093.347156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13093.358494] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13093.372686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13093.375504] Lustre: Skipped 1 previous similar message [13097.138526] Lustre: server umount lustre-MDT0000 complete [13100.007730] LustreError: 349001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13100.029977] LustreError: 349001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [13100.993395] LustreError: 348986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788435658 with bad export cookie 17324388682190126829 [13100.995767] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13101.003168] LustreError: 348986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13101.313683] Lustre: server umount lustre-MDT0001 complete [13110.030229] XFS (loop0): Unmounting Filesystem [13115.304731] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:41:11 (1788435671) [13122.893520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13123.337494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13123.342326] Lustre: Skipped 2 previous similar messages [13128.261921] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13138.331594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13143.910445] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13152.847324] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13159.406804] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13167.052407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13174.318485] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13182.802616] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13199.339125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13199.342059] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13199.358992] Lustre: Skipped 1 previous similar message [13199.360607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13199.366681] LustreError: Skipped 1 previous similar message [13199.391311] Lustre: Skipped 3 previous similar messages [13202.720297] Lustre: server umount lustre-OST0000 complete [13209.577516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13209.601235] Lustre: Skipped 1 previous similar message [13212.931585] Lustre: server umount lustre-MDT0000 complete [13216.111323] LustreError: 353504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788435773 with bad export cookie 17324388682190127753 [13216.123923] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13216.128033] LustreError: 353504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13216.430413] Lustre: server umount lustre-MDT0001 complete [13230.245900] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [13234.650369] Key type lgssc unregistered [13235.039602] LNet: 357155:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13235.056537] LNetError: 357155:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13236.150170] LNet: Removed LNI 192.168.202.101@tcp [13237.315194] Key type .llcrypt unregistered [13237.317034] Key type ._llcrypt unregistered [13268.616791] Key type ._llcrypt registered [13268.628482] Key type .llcrypt registered [13268.772144] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [13269.962852] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13270.183125] alg: No test for adler32 (adler32-zlib) [13271.490798] Lustre: Lustre: Build Version: 2.17.58_7_g065ac5e [13271.791661] LNet: Added LNI 192.168.202.101@tcp [8/256/0/180] [13273.463479] Key type lgssc registered [13275.081160] Lustre: Echo OBD driver; http://www.lustre.org/ [13283.134909] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13283.153697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13284.615276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13289.341413] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13295.080702] LustreError: 358720:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13295.103551] LustreError: 358720:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13298.590355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13298.949161] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13304.784439] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13314.539283] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13322.137181] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13329.912027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13330.179919] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13336.970794] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13345.829350] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13365.216640] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13365.249253] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13368.257933] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.1@tcp (stopping) [13368.266103] Lustre: Skipped 1 previous similar message [13370.345962] LustreError: 358721:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13370.357831] LustreError: 358721:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13370.370407] Lustre: server umount lustre-MDT0001 complete [13373.339042] LustreError: 358716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.1@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13375.461622] LustreError: 359661:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13375.485962] LustreError: 359661:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13380.598564] LustreError: 358720:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13380.635556] LustreError: 358720:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13382.939438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13383.181886] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13383.187467] Lustre: Found index 1 for lustre-MDT0001, updating log [13383.260774] Lustre: 358708:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13383.576086] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.1@tcp (not set up) [13383.791149] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13388.873573] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13389.292937] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13391.928259] LustreError: 362344:0:(obd_mount.c:312:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13391.933816] LustreError: 362344:0:(tgt_mount.c:739:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13391.943511] LustreError: 358713:0:(mgc_request.c:718:do_requeue()) failed processing log: -17 [13392.029099] LustreError: 362347:0:(obd_config.c:2100:class_config_llog_handler()) mgc192.168.202.101@tcp: cfg command failed: rc = -17 [13392.045990] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13392.045990] [13701.061506] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.1@tcp (stopping) [13701.601688] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13701.635306] Lustre: Skipped 1 previous similar message [13706.127471] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.1@tcp (stopping) [13706.140153] Lustre: Skipped 2 previous similar messages [13711.327187] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13711.856028] LustreError: 359661:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13711.873658] LustreError: 359661:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13712.170602] Lustre: server umount lustre-MDT0001 complete [13726.297448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13726.526398] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13726.534991] Lustre: Found index 1 for lustre-MDT0001, updating log [13726.628231] Lustre: 358707:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13726.976225] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13732.282980] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [13732.343634] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13735.490577] LustreError: 370090:0:(obd_mount.c:312:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13735.507489] LustreError: 370090:0:(tgt_mount.c:739:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13735.522119] LustreError: 358713:0:(mgc_request.c:718:do_requeue()) failed processing log: -17 [13735.528076] LustreError: 358713:0:(mgc_request.c:718:do_requeue()) Skipped 1 previous similar message [13735.645228] LustreError: 370098:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13735.660291] LustreError: 370098:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13735.669677] LustreError: 370098:0:(obd_config.c:2100:class_config_llog_handler()) mgc192.168.202.101@tcp: cfg command failed: rc = -17 [13735.682282] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.101@tcp [13735.682282] [14044.096538] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.1@tcp (stopping) [14044.113574] Lustre: Skipped 3 previous similar messages [14044.648608] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14044.649225] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14044.685058] Lustre: Skipped 1 previous similar message [14054.367226] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14054.653267] Lustre: server umount lustre-MDT0001 complete [14054.886412] LustreError: 358716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14054.896676] LustreError: 358716:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [14067.093795] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14067.258191] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14067.264694] Lustre: Found index 1 for lustre-MDT0001, updating log [14067.333414] Lustre: 358710:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14067.830040] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14072.703750] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [14073.340488] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14073.900802] LustreError: 377737:0:(obd_mount.c:312:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14073.918549] LustreError: 377737:0:(tgt_mount.c:739:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14073.942816] LustreError: 358713:0:(mgc_request.c:718:do_requeue()) failed processing log: -17 [14073.951758] LustreError: 358713:0:(mgc_request.c:718:do_requeue()) Skipped 1 previous similar message [14074.142915] LustreError: 377752:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14074.158840] LustreError: 377752:0:(obd_config.c:2100:class_config_llog_handler()) mgc192.168.202.101@tcp: cfg command failed: rc = -114 [14074.173762] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14074.173762] [14395.369441] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14395.385385] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14395.403945] Lustre: Skipped 1 previous similar message [14395.407288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14395.416654] Lustre: Skipped 6 previous similar messages [14399.756262] Lustre: server umount lustre-MDT0000 complete [14400.994174] LustreError: 358716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14401.011188] LustreError: 358716:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [14403.550610] LustreError: 358700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788436961 with bad export cookie 16021044530734498410 [14403.572053] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14403.578074] LustreError: 358700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14406.123580] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14406.154504] Lustre: Skipped 2 previous similar messages [14409.998868] Lustre: server umount lustre-MDT0001 complete [14414.718833] Lustre: server umount lustre-OST0000 complete [14424.100598] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [14435.450887] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [14472.763853] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:03:48 (1788437028) [14479.416357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14479.797938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14479.819622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14479.887479] Lustre: lustre-MDT0000: new disk, initializing [14479.976009] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14479.993209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14484.087613] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [14495.043870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14495.245317] Lustre: 388236:0:(mgs_llog.c:1456:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14495.397092] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14495.409204] Lustre: Skipped 1 previous similar message [14495.531212] Lustre: lustre-MDT0001: new disk, initializing [14495.641250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14495.679669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14495.693203] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14500.671375] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [14510.387196] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14516.155889] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14523.553374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14523.777277] Lustre: lustre-OST0000: new disk, initializing [14523.779690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14523.784691] Lustre: 390204:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14523.870123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14530.960343] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_default_debug -1 all [14533.807598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14533.826223] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14533.940253] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14541.773644] Lustre: DEBUG MARKER: oleg201-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14549.475527] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14549.483417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14549.509258] Lustre: Skipped 4 previous similar messages [14549.512045] Lustre: Skipped 1 previous similar message [14551.861789] Lustre: server umount lustre-MDT0000 complete [14554.594461] LustreError: 388243:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14554.603759] LustreError: 388243:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14555.234113] LustreError: 388230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788437113 with bad export cookie 16021044530734500503 [14555.243272] LustreError: mgc192.168.202.101@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14555.243796] LustreError: 388230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14555.264162] LustreError: 389043:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14555.531166] Lustre: server umount lustre-MDT0001 complete [14568.498601] Lustre: server umount lustre-OST0000 complete [14601.612107] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing set_hostid [14614.059828] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing load_modules_local [14649.845250] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14362 sec ========= 08:06:45 (1788437205) [14652.112925] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:06:47 (1788437207) === [14656.146554] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:06:51 (1788437211) === [14697.414655] Lustre: DEBUG MARKER: oleg201-server.virtnet: executing unload_modules_local [14701.135420] Key type lgssc unregistered [14701.577482] LNet: 396571:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14701.589162] LNetError: 396571:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14701.609863] LNet: Removed LNI 192.168.202.101@tcp [14702.822374] Key type .llcrypt unregistered [14702.827353] Key type ._llcrypt unregistered