[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 675737263 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001016] APIC: Switch to symmetric I/O mode setup [ 0.002106] x2apic enabled [ 0.003013] Switched APIC routing to physical x2apic. [ 0.005012] kvm-guest: setup PV IPIs [ 0.007587] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008000] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008018] pid_max: default: 32768 minimum: 301 [ 0.009000] LSM: Security Framework initializing [ 0.009081] Yama: becoming mindful. [ 0.010000] SELinux: Initializing. [ 0.011128] *** VALIDATE selinux *** [ 0.021026] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028085] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029180] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031139] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032148] *** VALIDATE tmpfs *** [ 0.033561] *** VALIDATE proc *** [ 0.034302] *** VALIDATE cgroup *** [ 0.035023] *** VALIDATE cgroup2 *** [ 0.037350] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038207] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039015] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040035] Spectre V2 : User space: Vulnerable [ 0.041020] Speculative Store Bypass: Vulnerable [ 0.046472] debug: unmapping init [mem 0xffffffff96259000-0xffffffff96260fff] [ 0.048383] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050375] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051032] ... version: 2 [ 0.052027] ... bit width: 48 [ 0.053028] ... generic registers: 4 [ 0.054021] ... value mask: 0000ffffffffffff [ 0.055031] ... max period: 00007fffffffffff [ 0.056031] ... fixed-purpose events: 3 [ 0.057026] ... event mask: 000000070000000f [ 0.058389] rcu: Hierarchical SRCU implementation. [ 0.061187] smp: Bringing up secondary CPUs ... [ 0.063110] x86: Booting SMP configuration: [ 0.064022] .... node #0, CPUs: #1 #2 #3 [ 0.083118] smp: Brought up 1 node, 4 CPUs [ 0.085015] smpboot: Max logical packages: 1 [ 0.086015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.174443] node 0 deferred pages initialised in 84ms [ 0.182151] devtmpfs: initialized [ 0.183356] x86/mm: Memory block size: 128MB [ 0.185860] gcov: version magic: 0x41383552 [ 0.187480] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.188114] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.199070] pinctrl core: initialized pinctrl subsystem [ 0.205451] [ 0.207010] ************************************************************* [ 0.212015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.219027] ** ** [ 0.223013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.233017] ** ** [ 0.238015] ** This means that this kernel is built to expose internal ** [ 0.244017] ** IOMMU data structures, which may compromise security on ** [ 0.251017] ** your system. ** [ 0.256015] ** ** [ 0.259015] ** If you see this message and you are not debugging the ** [ 0.262014] ** kernel, report this immediately to your vendor! ** [ 0.264014] ** ** [ 0.266015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.269015] ************************************************************* [ 0.276985] NET: Registered protocol family 16 [ 0.281737] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.285133] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.289068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.297137] cpuidle: using governor menu [ 0.307039] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.324169] PCI: Using configuration type 1 for base access [ 0.332800] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.362095] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.365022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.370088] cryptd: max_cpu_qlen set to 1000 [ 0.373254] ACPI: Added _OSI(Module Device) [ 0.375019] ACPI: Added _OSI(Processor Device) [ 0.376076] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.382068] ACPI: Added _OSI(Processor Aggregator Device) [ 0.392862] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.409563] ACPI: Interpreter enabled [ 0.410070] ACPI: PM: (supports S0 S3 S4 S5) [ 0.411000] ACPI: Using IOAPIC for interrupt routing [ 0.414160] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.419153] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.454352] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.459060] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.468021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.478090] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.495000] acpiphp: Slot [2] registered [ 0.497551] acpiphp: Slot [5] registered [ 0.498147] acpiphp: Slot [6] registered [ 0.499179] acpiphp: Slot [7] registered [ 0.501180] acpiphp: Slot [8] registered [ 0.504316] acpiphp: Slot [9] registered [ 0.506492] acpiphp: Slot [10] registered [ 0.507000] acpiphp: Slot [3] registered [ 0.511141] acpiphp: Slot [4] registered [ 0.512135] acpiphp: Slot [11] registered [ 0.515146] acpiphp: Slot [12] registered [ 0.518174] acpiphp: Slot [13] registered [ 0.523143] acpiphp: Slot [14] registered [ 0.526149] acpiphp: Slot [15] registered [ 0.530168] acpiphp: Slot [16] registered [ 0.532138] acpiphp: Slot [17] registered [ 0.535719] acpiphp: Slot [18] registered [ 0.538377] acpiphp: Slot [19] registered [ 0.541153] acpiphp: Slot [20] registered [ 0.544208] acpiphp: Slot [21] registered [ 0.546339] acpiphp: Slot [22] registered [ 0.550110] acpiphp: Slot [23] registered [ 0.553157] acpiphp: Slot [24] registered [ 0.556146] acpiphp: Slot [25] registered [ 0.559135] acpiphp: Slot [26] registered [ 0.560255] acpiphp: Slot [27] registered [ 0.564308] acpiphp: Slot [28] registered [ 0.566226] acpiphp: Slot [29] registered [ 0.569122] acpiphp: Slot [30] registered [ 0.572130] acpiphp: Slot [31] registered [ 0.577102] PCI host bridge to bus 0000:00 [ 0.579022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.581023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.587022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.592040] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.598034] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.601033] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.605413] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.614421] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.618394] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.634016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.642060] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.644020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.648020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.652019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.656783] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.659915] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.662049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.663990] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.670084] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.685017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.691017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.703882] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.724025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.735025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.781025] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.794300] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.804022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.811021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.826023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.835000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.840022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.845000] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.860020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.874000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.880021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.886020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.905023] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.926856] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.939021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.955019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.992023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.002068] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.020018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.042022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.077311] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.106417] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.112718] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.120793] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.125643] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.132843] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.153122] iommu: Default domain type: Passthrough [ 1.159491] SCSI subsystem initialized [ 1.169533] ACPI: bus type USB registered [ 1.172116] usbcore: registered new interface driver usbfs [ 1.178103] usbcore: registered new interface driver hub [ 1.182126] usbcore: registered new device driver usb [ 1.186201] pps_core: LinuxPPS API ver. 1 registered [ 1.189015] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.195066] PTP clock support registered [ 1.199377] EDAC MC: Ver: 3.0.0 [ 1.203172] PCI: Using ACPI for IRQ routing [ 1.208593] NetLabel: Initializing [ 1.211014] NetLabel: domain hash size = 128 [ 1.216012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.222775] NetLabel: unlabeled traffic allowed by default [ 1.225042] vgaarb: loaded [ 1.227331] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.231013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.241736] clocksource: Switched to clocksource kvm-clock [ 1.390158] VFS: Disk quotas dquot_6.6.0 [ 1.391806] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.394545] *** VALIDATE ramfs *** [ 1.395606] *** VALIDATE hugetlbfs *** [ 1.397605] pnp: PnP ACPI init [ 1.401564] pnp: PnP ACPI: found 6 devices [ 1.431513] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.437211] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.441754] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.444968] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.449049] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.453414] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.457795] NET: Registered protocol family 2 [ 1.467874] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.480743] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.495847] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.504821] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.509712] TCP: Hash tables configured (established 65536 bind 65536) [ 1.513819] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.519386] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.530867] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.542965] NET: Registered protocol family 1 [ 1.547718] RPC: Registered named UNIX socket transport module. [ 1.550678] RPC: Registered udp transport module. [ 1.552811] RPC: Registered tcp transport module. [ 1.555054] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.562061] NET: Registered protocol family 44 [ 1.563807] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.566482] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.574850] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.578397] PCI: CLS 0 bytes, default 64 [ 1.582037] Unpacking initramfs... [ 6.020302] debug: unmapping init [mem 0xffff8a0b7cc54000-0xffff8a0b7ffbffff] [ 6.045752] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 6.048494] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 6.057715] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 7.256800] Initialise system trusted keyrings [ 7.260572] Key type blacklist registered [ 7.263395] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 7.287200] zbud: loaded [ 7.299219] *** VALIDATE nfs *** [ 7.301074] *** VALIDATE nfs4 *** [ 7.303645] pstore: using deflate compression [ 7.320837] Platform Keyring initialized [ 7.664801] NET: Registered protocol family 38 [ 7.668293] Key type asymmetric registered [ 7.669571] Asymmetric key parser 'x509' registered [ 7.673384] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 7.680579] io scheduler mq-deadline registered [ 7.682021] io scheduler kyber registered [ 7.683185] io scheduler bfq registered [ 7.687926] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 7.692459] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 7.696731] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 7.701269] ACPI: Power Button [PWRF] [ 7.715187] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 7.729391] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 7.747207] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 7.758048] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 7.787142] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 7.833024] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 7.878714] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 7.894183] Non-volatile memory driver v1.3 [ 7.895921] Linux agpgart interface v0.103 [ 7.959750] virtio_blk virtio1: [vda] 149760 512-byte logical blocks (76.7 MB/73.1 MiB) [ 7.965523] vda: detected capacity change from 0 to 76677120 [ 7.988532] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 7.992369] vdb: detected capacity change from 0 to 1073741824 [ 8.022924] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 8.027909] vdc: detected capacity change from 0 to 2621440000 [ 8.057046] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 8.060879] vdd: detected capacity change from 0 to 2621440000 [ 8.108547] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 8.117111] vde: detected capacity change from 0 to 4294967296 [ 8.151735] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 8.160837] vdf: detected capacity change from 0 to 4294967296 [ 8.177554] libphy: Fixed MDIO Bus: probed [ 8.192939] usbcore: registered new interface driver usbserial_generic [ 8.195744] usbserial: USB Serial support registered for generic [ 8.198299] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 8.203845] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 8.205907] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 8.209236] mousedev: PS/2 mouse device common for all mice [ 8.213979] rtc_cmos 00:05: RTC can wake from S4 [ 8.218341] rtc_cmos 00:05: registered as rtc0 [ 8.222394] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 8.225910] intel_pstate: CPU model not supported [ 8.228328] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 8.232072] hid: raw HID events driver (C) Jiri Kosina [ 8.234608] usbcore: registered new interface driver usbhid [ 8.238088] usbhid: USB HID core driver [ 8.241035] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 8.241833] drop_monitor: Initializing network drop monitor service [ 8.241958] Initializing XFRM netlink socket [ 8.244232] NET: Registered protocol family 10 [ 8.257671] Segment Routing with IPv6 [ 8.259183] NET: Registered protocol family 17 [ 8.260493] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 8.266773] mpls_gso: MPLS GSO support [ 8.277668] RAS: Correctable Errors collector initialized. [ 8.280609] AVX version of gcm_enc/dec engaged. [ 8.282874] AES CTR mode by8 optimization enabled [ 8.388579] sched_clock: Marking stable (8388372048, 0)->(10224204461, -1835832413) [ 8.393638] registered taskstats version 1 [ 8.395992] Loading compiled-in X.509 certificates [ 8.398217] zswap: loaded using pool lzo/zbud [ 8.431769] Key type big_key registered [ 8.448131] Key type encrypted registered [ 8.456395] ima: No TPM chip found, activating TPM-bypass! [ 8.464059] ima: Allocated hash algorithm: sha1 [ 8.472411] ima: No architecture policies found [ 8.478253] evm: Initialising EVM extended attributes: [ 8.480038] evm: security.selinux [ 8.484422] evm: security.ima [ 8.486649] evm: security.capability [ 8.488979] evm: HMAC attrs: 0x1 [ 8.499312] rtc_cmos 00:05: setting system clock to 2026-09-05 14:29:24 UTC (1788618564) [ 8.515119] debug: unmapping init [mem 0xffffffff97203000-0xffffffff973fffff] [ 8.526874] debug: unmapping init [mem 0xffffffff95f82000-0xffffffff96258fff] [ 8.537224] Write protecting the kernel read-only data: 28672k [ 8.542927] debug: unmapping init [mem 0xffffffff94603000-0xffffffff947fffff] [ 8.554982] debug: unmapping init [mem 0xffffffff94f14000-0xffffffff94ffffff] [ 8.653792] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 8.664178] systemd[1]: Detected virtualization kvm. [ 8.676624] systemd[1]: Detected architecture x86-64. [ 8.685031] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 8.750451] systemd[1]: No hostname configured. [ 8.754747] systemd[1]: Set hostname to . [ 8.759632] random: systemd: uninitialized urandom read (16 bytes read) [ 8.771386] systemd[1]: Initializing machine ID from random generator. [ 8.942671] random: ln: uninitialized urandom read (6 bytes read) [ 9.095449] random: systemd: uninitialized urandom read (16 bytes read) [ 9.099466] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 9.108220] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 9.117727] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Slices. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 10.357473] device-mapper: uevent: version 1.0.3 [ 10.374971] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon[ 11.951603] virtio_net virtio0 ens2: renamed from eth0 . Starting dracut initqueue hook... [ 12.081855] random: fast init done [ 12.376517] scsi host0: ata_piix [ 12.417088] scsi host1: ata_piix [ 12.418854] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.421677] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 17.012245] random: crng init done [ 17.013905] random: 7 urandom warning(s) missed due to ratelimiting [ 17.186450] dracut-initqueue[583]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 18.966139] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.026637] printk: systemd: 26 output lines suppressed due to ratelimiting [ 21.754660] SELinux: Disabled at runtime. [ 21.840653] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 21.864616] systemd[1]: Detected virtualization kvm. [ 21.877045] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 23.320413] systemd[1]: initrd-switch-root.service: Succeeded. [ 23.324759] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 23.336718] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.346533] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.352732] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.368442] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.380500] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. Mounting POSIX Message Queue File System... [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ 23.859110] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 24.426343] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 25.220106] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 25.255398] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 25.697930] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 25.749840] EDAC sbridge: Ver: 1.1.2 [ 29.952357] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit)[ 30.555593] NFS: Registering the id_resolver key type [ 30.559565] Key type id_resolver registered [ 30.561766] Key type id_legacy registered [** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg638-server login: [ 86.278129] libcfs: loading out-of-tree module taints kernel. [ 86.315886] Key type ._llcrypt registered [ 86.318821] Key type .llcrypt registered [ 86.418423] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 104.096340] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 105.262287] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 105.274902] alg: No test for adler32 (adler32-zlib) [ 106.741446] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 107.497472] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 109.223183] Key type lgssc registered [ 110.816496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 130.663861] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 174.447029] hrtimer: interrupt took 2037868 ns [ 177.911131] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 192.193110] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 192.236289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.490055] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 193.538722] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.686459] Lustre: lustre-MDT0000: new disk, initializing [ 193.848115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.871129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 199.070755] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 214.171051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.292294] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 214.366074] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 214.373560] Lustre: Skipped 1 previous similar message [ 214.475252] Lustre: lustre-MDT0001: new disk, initializing [ 214.543454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 214.574194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 214.595087] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 219.155514] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 223.588769] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 232.949494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 233.273579] Lustre: lustre-OST0000: new disk, initializing [ 233.282090] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 233.293201] Lustre: 8453:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 233.370045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 233.580586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 233.598776] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 233.639658] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 239.240179] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 253.266490] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 253.390783] Lustre: lustre-OST0001: new disk, initializing [ 253.394188] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 253.398701] Lustre: 9525:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 253.458054] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 259.703969] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 262.175603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 262.181637] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 262.255909] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 270.819179] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 275.963802] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 282.311150] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing check_logdir /tmp/testlogs/ [ 287.438878] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing yml_node [ 291.286890] Lustre: DEBUG MARKER: Client: 2.17.58.3 [ 293.536467] Lustre: DEBUG MARKER: MDS: 2.17.58.3 [ 295.942924] Lustre: DEBUG MARKER: OSS: 2.17.58.3 [ 297.585286] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Sep 5 10:34:11 EDT 2026 [ 314.775662] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 316.398430] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 333.793342] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 333.800250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 333.823354] Lustre: Skipped 3 previous similar messages [ 333.838559] Lustre: Skipped 3 previous similar messages [ 338.912473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 338.924555] Lustre: Skipped 3 previous similar messages [ 340.020843] Lustre: server umount lustre-MDT0000 complete [ 343.522863] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788618899 with bad export cookie 18155639754171645933 [ 343.527547] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 343.531821] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 344.039167] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 344.040675] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 344.070884] Lustre: Skipped 1 previous similar message [ 348.064531] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 348.073343] Lustre: Skipped 2 previous similar messages [ 349.817635] Lustre: server umount lustre-MDT0001 complete [ 354.044493] Lustre: server umount lustre-OST0000 complete [ 357.607791] Lustre: server umount lustre-OST0001 complete [ 364.367062] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 374.823563] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 412.937895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.230956] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 413.272407] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 413.353039] Lustre: lustre-MDT0000: new disk, initializing [ 413.429259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 413.443061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 417.343623] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 428.428600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 428.607316] Lustre: 15077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 428.670439] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 428.679348] Lustre: Skipped 1 previous similar message [ 428.824846] Lustre: lustre-MDT0001: new disk, initializing [ 428.970883] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 429.036306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 429.055023] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 434.309644] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 444.528542] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 449.203860] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 455.351122] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 455.660989] Lustre: lustre-OST0000: new disk, initializing [ 455.665596] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 455.674911] Lustre: 17047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 455.796345] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 457.137640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 457.153439] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 457.311067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 462.235919] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 471.174063] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 475.638331] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 475.880765] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 479.941938] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 480.112599] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 482.788518] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 482.793094] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 482.793155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 482.806276] Lustre: Skipped 1 previous similar message [ 493.029117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 493.047718] Lustre: Skipped 4 previous similar messages [ 496.096583] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 496.321570] Lustre: server umount lustre-OST0000 complete [ 500.706765] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 500.723109] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 506.526991] Lustre: server umount lustre-MDT0000 complete [ 508.388634] LustreError: 15090:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 508.400567] LustreError: 15090:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 510.042892] LustreError: 17046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619066 with bad export cookie 18155639754171647466 [ 510.046362] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 510.053854] LustreError: 17046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 510.395570] Lustre: server umount lustre-MDT0001 complete [ 516.910250] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:37:50 (1788619070) [ 523.752244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 524.265908] LustreError: 19313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.349464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 529.389083] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 529.398726] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 534.514587] LustreError: 19313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.969942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.296534] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 550.088053] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 555.498584] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 561.746830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 568.853291] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 577.181917] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 582.202407] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 594.918024] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 594.922421] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 594.926659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 594.926667] Lustre: Skipped 3 previous similar messages [ 594.939876] Lustre: Skipped 3 previous similar messages [ 596.585081] Lustre: server umount lustre-MDT0000 complete [ 599.936785] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619155 with bad export cookie 18155639754171648348 [ 599.938388] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 599.944697] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 600.034780] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.185170] Lustre: server umount lustre-MDT0001 complete [ 606.877742] Lustre: server umount lustre-OST0000 complete [ 628.209771] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 637.754750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.129057] LustreError: 23690:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 638.187061] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 638.192676] Lustre: Skipped 2 previous similar messages [ 642.099208] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 650.409612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 654.835530] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 657.539516] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 663.386534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 669.382801] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 677.804452] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 677.960359] Lustre: lustre-OST0001: new disk, initializing [ 677.963445] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 677.968679] Lustre: 26007:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 679.732135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 679.737020] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 679.884242] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 684.180966] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 694.311258] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 698.071385] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 712.163572] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 712.172339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 712.180152] Lustre: Skipped 1 previous similar message [ 712.186167] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 712.190560] Lustre: Skipped 3 previous similar messages [ 716.862366] Lustre: server umount lustre-MDT0000 complete [ 719.848791] LustreError: 23686:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 719.869686] LustreError: 23686:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 720.627415] LustreError: 24430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619276 with bad export cookie 18155639754171649195 [ 720.635272] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 720.646716] LustreError: 24430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 721.058288] Lustre: server umount lustre-MDT0001 complete [ 734.299419] Lustre: server umount lustre-OST0000 complete [ 749.201806] Lustre: server umount lustre-OST0001 complete [ 767.134820] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 775.855404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 776.129292] LustreError: 29027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 776.218845] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 776.236230] Lustre: Skipped 3 previous similar messages [ 780.643559] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 790.205078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 795.526067] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 798.875780] Lustre: 30174:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 805.835466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 812.638765] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 821.655437] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.264827] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 835.323213] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 838.634761] Lustre: 32055:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 846.816595] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 846.826597] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 846.847650] Lustre: Skipped 3 previous similar messages [ 846.853573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 846.858993] Lustre: Skipped 4 previous similar messages [ 850.287436] Lustre: server umount lustre-MDT0000 complete [ 852.962177] LustreError: 29027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 852.975553] LustreError: 29027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 854.056033] LustreError: 31367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619410 with bad export cookie 18155639754171650490 [ 854.064050] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 854.073877] LustreError: 31367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 854.502696] Lustre: server umount lustre-MDT0001 complete [ 868.537596] Lustre: server umount lustre-OST0000 complete [ 882.773917] Lustre: server umount lustre-OST0001 complete [ 892.471994] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:44:06 (1788619446) [ 920.773833] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 930.249658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 935.562283] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 944.786323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 950.438254] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 953.322430] Lustre: 36005:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 960.267948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 966.808488] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 975.353192] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 982.831792] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 990.570088] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 994.431438] Lustre: 37885:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1001.441797] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1001.459558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1001.464219] Lustre: Skipped 5 previous similar messages [ 1001.475910] Lustre: Skipped 7 previous similar messages [ 1005.432420] Lustre: server umount lustre-MDT0000 complete [ 1006.572149] LustreError: 34854:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1006.612229] LustreError: 34854:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 1010.392342] LustreError: 34839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619566 with bad export cookie 18155639754171651596 [ 1010.392342] LustreError: 36008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619566 with bad export cookie 18155639754171651596 [ 1010.406503] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1010.406729] LustreError: 36008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1010.689633] Lustre: server umount lustre-MDT0001 complete [ 1024.157976] Lustre: server umount lustre-OST0000 complete [ 1027.111683] Lustre: 3651:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788619567/real 1788619567] req@ffff8a0bf772ed80 x1875502603510912/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788619583 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1028.217472] Lustre: server umount lustre-OST0001 complete [ 1036.780496] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:46:30 (1788619590) [ 1051.909789] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1061.295212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1061.769861] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1061.782815] Lustre: Skipped 7 previous similar messages [ 1065.853438] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1074.520538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1079.966775] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1082.917769] Lustre: 41177:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1089.223167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1096.564758] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1106.569653] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1114.259274] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1122.083341] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1125.714192] Lustre: 43055:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1126.596696] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1126.601982] Lustre: Skipped 1 previous similar message [ 1131.491328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1131.500592] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1131.519330] Lustre: Skipped 2 previous similar messages [ 1135.642785] Lustre: server umount lustre-MDT0000 complete [ 1138.958039] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619694 with bad export cookie 18155639754171652702 [ 1138.970490] LustreError: 40009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1138.973582] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1145.360724] Lustre: server umount lustre-MDT0001 complete [ 1154.730561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1170.108474] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1173.289454] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1178.427131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.215308] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1183.222154] Lustre: Skipped 1 previous similar message [ 1183.445559] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1190.319565] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1191.560969] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1197.304656] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1199.039622] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1204.704671] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1212.308856] Lustre: server umount lustre-MDT0000 complete [ 1215.214832] LustreError: 40062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619771 with bad export cookie 18155639754171653829 [ 1215.220510] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1215.224766] LustreError: 40062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1215.563292] Lustre: server umount lustre-MDT0001 complete [ 1228.386109] Lustre: server umount lustre-OST0000 complete [ 1242.617444] Lustre: server umount lustre-OST0001 complete [ 1250.661320] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:50:04 (1788619804) [ 1266.616772] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1276.457867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1277.191280] LustreError: 47658:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1277.218698] LustreError: 47658:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 24 previous similar messages [ 1282.133811] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1291.527054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1297.084058] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1300.215548] Lustre: 48810:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1306.941590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1314.019765] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1323.036112] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1329.612985] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1337.670177] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1342.462696] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1358.314076] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1358.325316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1358.340802] Lustre: Skipped 9 previous similar messages [ 1358.350040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1358.358952] Lustre: Skipped 14 previous similar messages [ 1361.823232] Lustre: server umount lustre-MDT0000 complete [ 1365.434287] LustreError: 50766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788619921 with bad export cookie 18155639754171654620 [ 1365.435641] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1365.446983] LustreError: 50766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1365.946827] Lustre: server umount lustre-MDT0001 complete [ 1379.464764] Lustre: server umount lustre-OST0000 complete [ 1394.234144] Lustre: server umount lustre-OST0001 complete [ 1403.967609] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:52:37 (1788619957) [ 1406.755245] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1408.851656] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:52:42 (1788619962) [ 1419.880925] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 1428.315241] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1453.980754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1454.270925] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1454.367129] Lustre: lustre-MDT0000: new disk, initializing [ 1454.469370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1460.089234] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1470.259968] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1477.740051] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1477.983359] Lustre: lustre-OST0000: new disk, initializing [ 1477.990864] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1477.996786] Lustre: Skipped 1 previous similar message [ 1478.004498] Lustre: 54890:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1480.082055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1480.091145] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1480.123278] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1484.403753] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1493.888886] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1496.422603] Lustre: DEBUG MARKER: create test files [ 1526.246135] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1531.280433] Lustre: server umount lustre-OST0000 complete [ 1536.088365] Lustre: server umount lustre-MDT0000 complete [ 1547.105563] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 1550.370493] Key type lgssc unregistered [ 1550.810280] LNet: 56518:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1550.825182] LNetError: 56518:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1550.852535] LNet: Removed LNI 192.168.206.138@tcp [ 1551.872513] Key type .llcrypt unregistered [ 1551.875505] Key type ._llcrypt unregistered [ 1579.608838] Key type ._llcrypt registered [ 1579.611204] Key type .llcrypt registered [ 1579.713460] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1580.765600] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1580.800929] alg: No test for adler32 (adler32-zlib) [ 1582.029291] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 1582.359429] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 1583.999170] Key type lgssc registered [ 1584.889823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1592.760609] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1592.791851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1594.392867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1599.137817] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1607.906633] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1615.130671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1615.530638] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1619.644221] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:96 to 0x240000400:129) [ 1619.813835] Lustre: 59228:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1619.827392] Lustre: 59228:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1619.840895] Lustre: 59228:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1619.853726] Lustre: 59228:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1619.867175] Lustre: 59228:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1619.874021] Lustre: 59228:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1623.466681] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1633.838605] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1639.731979] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1646.024252] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1655.787146] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1655.796822] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1655.830523] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1660.784219] Lustre: server umount lustre-OST0000 complete [ 1666.392415] Lustre: server umount lustre-MDT0000 complete [ 1678.809971] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 1681.515727] Key type lgssc unregistered [ 1681.773248] LNet: 60838:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1681.783753] LNetError: 60838:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1681.795945] LNet: Removed LNI 192.168.206.138@tcp [ 1682.515264] Key type .llcrypt unregistered [ 1682.518990] Key type ._llcrypt unregistered [ 1709.462510] Key type ._llcrypt registered [ 1709.465621] Key type .llcrypt registered [ 1709.572486] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1710.819314] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1710.863040] alg: No test for adler32 (adler32-zlib) [ 1711.953395] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 1712.166701] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 1713.823610] Key type lgssc registered [ 1714.930020] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1722.984588] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1723.025797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1724.679361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.735631] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1737.338698] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1744.165463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1744.584824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1749.755260] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1749.850203] Lustre: 63649:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1749.862777] Lustre: 63649:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1749.868435] Lustre: 63649:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1749.877333] Lustre: 63649:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1749.882099] Lustre: 63649:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1749.887585] Lustre: 63649:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1750.993613] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1758.659401] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1765.286895] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1775.584814] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1775.603813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1775.620487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1779.771637] Lustre: server umount lustre-OST0000 complete [ 1783.541251] Lustre: server umount lustre-MDT0000 complete [ 1794.710372] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 1797.541730] Key type lgssc unregistered [ 1797.894637] LNet: 65207:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1797.904211] LNetError: 65207:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1797.914794] LNet: Removed LNI 192.168.206.138@tcp [ 1798.786831] Key type .llcrypt unregistered [ 1798.789667] Key type ._llcrypt unregistered [ 1816.741722] Key type ._llcrypt registered [ 1816.749994] Key type .llcrypt registered [ 1816.854491] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 1831.401332] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 1832.311563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1832.438214] alg: No test for adler32 (adler32-zlib) [ 1833.634828] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 1833.977473] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 1835.607165] Key type lgssc registered [ 1836.759142] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1878.344876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1878.370211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1879.628639] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1879.668380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1879.776596] Lustre: lustre-MDT0000: new disk, initializing [ 1879.874607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1879.892401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1883.853736] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1893.779677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1893.908410] Lustre: 68358:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1893.962915] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1893.966138] Lustre: Skipped 1 previous similar message [ 1894.046424] Lustre: lustre-MDT0001: new disk, initializing [ 1894.146704] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1894.219110] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1894.245375] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1898.833690] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1908.552981] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1914.493701] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1921.579595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1921.894874] Lustre: lustre-OST0000: new disk, initializing [ 1921.899040] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1921.905491] Lustre: 70337:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1922.003552] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1928.783509] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 1931.830554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1931.840652] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1931.990645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1938.639189] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1943.095554] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1943.277897] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1947.720247] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1947.894687] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1952.737135] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1952.746026] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1952.754020] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1952.757616] Lustre: Skipped 1 previous similar message [ 1956.290729] Lustre: server umount lustre-OST0000 complete [ 1960.931129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1960.942904] LustreError: Skipped 1 previous similar message [ 1960.951723] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1960.975470] Lustre: Skipped 1 previous similar message [ 1960.978105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1962.977241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1962.981046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1963.006194] Lustre: Skipped 1 previous similar message [ 1966.604933] Lustre: server umount lustre-MDT0000 complete [ 1970.642848] LustreError: 68350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788620526 with bad export cookie 18326781584976202795 [ 1970.649640] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1970.666109] LustreError: 68350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1971.197598] Lustre: server umount lustre-MDT0001 complete [ 1980.264857] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:02:14 (1788620534) [ 1990.674953] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 2031.979306] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 2042.004290] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 2081.976821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2082.261505] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2082.281245] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2082.331581] Lustre: lustre-MDT0000: new disk, initializing [ 2082.385715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2082.396484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2087.607788] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2100.450615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2100.691470] Lustre: 76358:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2100.823139] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2100.845149] Lustre: Skipped 1 previous similar message [ 2101.051768] Lustre: lustre-MDT0001: new disk, initializing [ 2101.251138] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2101.385896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2101.393990] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2105.837118] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2116.303686] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2121.487934] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2128.134598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2128.389078] Lustre: lustre-OST0000: new disk, initializing [ 2128.393699] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2128.400844] Lustre: 78326:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2128.494572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2134.376885] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2135.621930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2135.631197] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2135.726402] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2143.944737] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2148.818967] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2149.077612] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2153.467808] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2153.638962] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2156.514270] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2156.526842] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2156.528545] LustreError: Skipped 1 previous similar message [ 2156.529382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2156.552472] Lustre: Skipped 1 previous similar message [ 2161.609197] Lustre: server umount lustre-OST0000 complete [ 2166.753677] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2166.776662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2166.789128] Lustre: Skipped 2 previous similar messages [ 2171.591826] Lustre: server umount lustre-MDT0000 complete [ 2171.871983] LustreError: 76366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2171.896230] LustreError: 76366:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2175.282793] LustreError: 78337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788620731 with bad export cookie 18326781584976203677 [ 2175.287226] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2175.299865] LustreError: 78337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2175.623785] Lustre: server umount lustre-MDT0001 complete [ 2184.495673] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 11:05:38 (1788620738) [ 2192.770102] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2193.223510] LustreError: 80741:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2193.314582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2198.504710] LustreError: 80742:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2198.562988] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2203.623724] LustreError: 80741:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2207.977764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2213.693310] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2221.832967] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2227.846596] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2235.885714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2236.268459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2236.276778] Lustre: Skipped 1 previous similar message [ 2243.383906] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2251.268685] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2255.196582] Lustre: 83412:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 67bc6af8-0100-47b7-af75-0a70c1b9af38 at adminstrative request [ 2258.933266] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2258.947251] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2268.129538] Lustre: MGS: Client 67bc6af8-0100-47b7-af75-0a70c1b9af38 (at 0@lo) reconnecting [ 2268.149986] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xfe55cec088ad5308 to 0xfe55cec088ad55af [ 2268.174916] Lustre: MGC192.168.206.138@tcp: Connection restored to 0@lo (at 0@lo) [ 2294.406822] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2294.595223] Lustre: lustre-OST0001: new disk, initializing [ 2294.605846] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2294.616601] Lustre: 83805:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2300.969732] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2302.646379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2302.658911] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2302.808751] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2311.671308] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2318.305713] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2318.311826] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2318.315903] LustreError: Skipped 1 previous similar message [ 2318.327587] Lustre: Skipped 1 previous similar message [ 2318.329114] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2318.336828] Lustre: Skipped 1 previous similar message [ 2322.312636] Lustre: server umount lustre-OST0001 complete [ 2323.431125] LustreError: 83446:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2323.461083] LustreError: 83446:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2333.653752] Lustre: server umount lustre-OST0000 complete [ 2338.785815] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2338.802666] Lustre: Skipped 3 previous similar messages [ 2338.811382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2338.821271] Lustre: Skipped 3 previous similar messages [ 2344.070167] Lustre: server umount lustre-MDT0000 complete [ 2348.241705] LustreError: 80722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788620904 with bad export cookie 18326781584976205231 [ 2348.250023] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2348.258925] LustreError: 80722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2348.635942] Lustre: server umount lustre-MDT0001 complete [ 2359.090407] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 11:08:32 (1788620912) [ 2360.828263] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2362.883347] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 11:08:36 (1788620916) [ 2364.653421] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2366.715627] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 11:08:40 (1788620920) [ 2368.461584] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2370.293436] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 11:08:44 (1788620924) [ 2371.891992] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2373.555153] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 11:08:47 (1788620927) [ 2375.547788] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2382.049518] loop: module loaded [ 2382.081035] loop0: detected capacity change from 0 to 4294967296 [ 2382.143170] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2383.965310] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2387.727979] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2391.436629] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2420.227335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2420.256355] LustreError: 87931:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2420.266404] LustreError: 87931:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2420.274352] LustreError: 87931:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2420.285384] LustreError: 87931:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2440.846643] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 2451.667658] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 2491.233518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2491.544113] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2491.574617] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2491.721739] Lustre: lustre-MDT0000: new disk, initializing [ 2491.881053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2491.892744] Lustre: Skipped 1 previous similar message [ 2491.924703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2497.119618] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2509.325660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2509.471646] Lustre: 90798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2509.924664] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2516.235761] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2526.222937] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2532.261209] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2538.920193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2539.189799] Lustre: 92769:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2545.227352] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2545.372819] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2545.552264] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2556.430649] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2561.884737] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2562.145732] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2567.492196] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2567.681366] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2571.233838] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2571.241994] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2571.243934] LustreError: Skipped 1 previous similar message [ 2571.245634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2571.270629] Lustre: Skipped 2 previous similar messages [ 2571.273489] Lustre: Skipped 2 previous similar messages [ 2575.910492] Lustre: server umount lustre-OST0000 complete [ 2581.472353] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2585.957223] Lustre: server umount lustre-MDT0000 complete [ 2589.669740] LustreError: 91889:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2589.682479] LustreError: 91889:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2589.719640] LustreError: 90789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788621145 with bad export cookie 18326781584976205847 [ 2589.738912] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2590.139378] Lustre: server umount lustre-MDT0001 complete [ 2599.101820] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 11:12:32 (1788621152) [ 2606.918343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2607.337415] LustreError: 95181:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2612.377283] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2621.651800] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2621.885929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2621.891337] Lustre: Skipped 3 previous similar messages [ 2625.820137] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2634.644592] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2641.446856] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2649.083145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2656.079872] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2663.405851] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2667.739972] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2668.046936] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2673.338548] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2673.504403] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2679.819636] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2679.979997] Lustre: lustre-OST0001: new disk, initializing [ 2679.982758] Lustre: Skipped 2 previous similar messages [ 2679.985808] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2679.989402] Lustre: Skipped 3 previous similar messages [ 2679.993769] Lustre: 98550:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2686.071821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2686.085219] Lustre: Skipped 2 previous similar messages [ 2686.089614] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2686.132319] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2686.204349] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2696.513578] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2701.071295] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2701.221255] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2705.044745] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2705.341348] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2726.565891] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2753.167541] Lustre: Failing over lustre-MDT0000 [ 2753.704092] Lustre: server umount lustre-MDT0000 complete [ 2754.299926] LustreError: 100452:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2754.327666] LustreError: 100452:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2755.040164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2755.052754] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2755.068883] Lustre: Skipped 2 previous similar messages [ 2774.504909] Lustre: 66579:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788621314/real 1788621314] req@000000008b024c16 x1875504413292160/t0(0) o400->MGC192.168.206.138@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788621330 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2774.539939] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2778.180148] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2778.182429] LDISKFS-fs (dm-0): recovery complete [ 2778.188462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2785.050563] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2785.975623] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2785.975945] Lustre: 95177:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2788.887373] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2790.373212] Lustre: 95181:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2790.379179] Lustre: 95181:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2790.394297] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2790.482833] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2790.807164] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2791.140528] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2791.150202] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2791.471128] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2791.474906] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2791.483553] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2791.500892] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2792.145991] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2792.158383] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2792.487223] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2792.497650] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2793.778632] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2793.803390] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2794.448171] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2794.459143] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2794.783173] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2794.790432] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2797.952797] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2797.959481] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2798.601138] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2798.612834] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2798.943276] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2798.951730] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2806.240247] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2806.260621] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2806.905228] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2806.920279] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2807.255292] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2807.262294] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2822.519263] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2822.535025] Lustre: 101398:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2823.178623] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2823.201528] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2823.527528] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2823.537475] LustreError: 101398:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2845.071188] Lustre: 101398:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2845.079797] Lustre: 101398:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2845.089064] Lustre: 101398:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 047895e2-fcb5-4dae-b569-8d64569c3689@192.168.206.38@tcp [ 2845.098295] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2845.102509] Lustre: 101398:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000e44cd4a5 x1875504398404480/t0(8589934768) o35->047895e2-fcb5-4dae-b569-8d64569c3689@192.168.206.38@tcp:741/0 lens 392/0 e 0 to 0 dl 1788621411 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2845.125602] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2845.139186] Lustre: lustre-MDT0000: Denying connection for new client 047895e2-fcb5-4dae-b569-8d64569c3689 (at 192.168.206.38@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2845.158950] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2845.184190] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2845.190393] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2845.198864] Lustre: Skipped 2 previous similar messages [ 2845.225529] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2845.225975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2860.513687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2860.522063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2860.529323] Lustre: Skipped 6 previous similar messages [ 2874.335721] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2874.453229] Lustre: server umount lustre-OST0000 complete [ 2877.417390] LustreError: 97004:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2877.431702] LustreError: 97004:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 42 previous similar messages [ 2884.273562] Lustre: server umount lustre-OST0001 complete [ 2891.234815] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2891.247992] Lustre: Skipped 7 previous similar messages [ 2894.020976] Lustre: server umount lustre-MDT0000 complete [ 2897.990251] LustreError: 95917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788621453 with bad export cookie 18326781584976257066 [ 2897.997655] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2898.004071] LustreError: 95917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2898.404151] Lustre: server umount lustre-MDT0001 complete [ 2907.625927] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 11:17:41 (1788621461) [ 2915.171433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2915.176756] Lustre: *** cfs_fail_loc=197, val=0*** [ 2930.719607] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2935.845070] LustreError: 103377:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.138@tcp: failed processing log, type 4: rc = -110 [ 2940.992702] Lustre: *** cfs_fail_loc=197, val=0*** [ 2940.996509] Lustre: Skipped 65 previous similar messages [ 2961.569081] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2961.576074] Lustre: Skipped 3 previous similar messages [ 2967.414437] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 2975.118517] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2976.940563] Lustre: server umount lustre-OST0000 complete [ 2985.183240] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 11:18:58 (1788621538) [ 3006.218425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3006.764202] LustreError: 105002:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3006.775653] LustreError: 105002:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3011.879541] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3022.057405] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3026.043554] LustreError: 105033:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3026.052323] LustreError: 105033:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3026.064644] LustreError: 105033:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3026.380063] Lustre: server umount lustre-MDT0000 complete [ 3061.841187] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 3071.178471] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 3104.224469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3104.481918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3104.518573] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3104.660088] Lustre: lustre-MDT0000: new disk, initializing [ 3104.734271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3109.298427] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3120.154932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3120.259900] Lustre: 109207:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3120.483596] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3124.557967] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3134.498956] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3140.297433] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3149.246788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3149.619884] Lustre: 111174:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3155.682863] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3156.927838] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3167.753342] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3173.866513] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3174.176163] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3179.969911] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3180.321563] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3186.658405] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3186.661708] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3186.673454] Lustre: Skipped 2 previous similar messages [ 3186.689960] Lustre: Skipped 15 previous similar messages [ 3188.705476] Lustre: server umount lustre-OST0000 complete [ 3199.810414] Lustre: server umount lustre-MDT0000 complete [ 3204.593134] LustreError: 109197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788621760 with bad export cookie 18326781584976267636 [ 3204.602605] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3204.606598] LustreError: 109197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3205.024605] Lustre: server umount lustre-MDT0001 complete [ 3214.541715] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 11:22:48 (1788621768) [ 3223.884878] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 3226.512330] Key type lgssc unregistered [ 3226.829046] LNet: 113664:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3226.835414] LNetError: 113664:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3226.856734] LNet: Removed LNI 192.168.206.138@tcp [ 3227.771146] Key type .llcrypt unregistered [ 3227.775719] Key type ._llcrypt unregistered [ 3244.898563] Key type ._llcrypt registered [ 3244.901289] Key type .llcrypt registered [ 3245.061390] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 3259.473551] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 3260.391868] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3260.447513] alg: No test for adler32 (adler32-zlib) [ 3261.495248] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 3261.706560] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 3263.441539] Key type lgssc registered [ 3264.547583] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3304.943903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3304.957628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3306.192519] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3306.239436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3306.347397] Lustre: lustre-MDT0000: new disk, initializing [ 3306.439701] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3306.461893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3310.425300] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3321.835034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3322.151697] Lustre: lustre-OST0000: new disk, initializing [ 3322.162778] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3322.169144] Lustre: Skipped 1 previous similar message [ 3322.179708] Lustre: 117834:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3322.303175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3327.499768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3327.509949] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3327.540941] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3329.338299] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3339.894478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3340.052285] Lustre: lustre-OST0001: new disk, initializing [ 3340.058381] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3340.069365] Lustre: 118697:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3340.146712] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3345.611359] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3349.162652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3349.173489] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3349.208893] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3354.594884] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3354.624465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3354.633060] Lustre: Skipped 1 previous similar message [ 3359.718829] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3359.723325] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3359.872797] Lustre: server umount lustre-MDT0000 complete [ 3366.894704] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3371.802150] Lustre: server umount lustre-OST0000 complete [ 3375.891988] Lustre: server umount lustre-OST0001 complete [ 3384.518990] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 11:25:38 (1788621938) [ 3407.624171] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 11:26:01 (1788621961) [ 3416.345891] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 3434.337160] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:26:28 (1788621988) [ 3472.415489] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 3481.537416] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 3517.392219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3517.620202] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3517.664905] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3517.733284] Lustre: lustre-MDT0000: new disk, initializing [ 3517.843125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3517.855661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3521.843612] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3532.455028] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3532.581138] Lustre: 126447:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3532.639778] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3532.642987] Lustre: Skipped 1 previous similar message [ 3532.735473] Lustre: lustre-MDT0001: new disk, initializing [ 3532.842765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3532.869079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3532.880786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3537.279761] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3546.192674] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3551.649380] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3558.579291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3558.915541] Lustre: lustre-OST0000: new disk, initializing [ 3558.920287] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3558.926704] Lustre: 128417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3559.027430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3564.937351] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3568.727487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3568.750865] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3568.867160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3575.294161] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3580.148893] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3580.371693] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3585.360178] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3585.535465] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3589.603335] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3589.609133] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3589.617749] Lustre: Skipped 1 previous similar message [ 3589.623066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3589.632366] Lustre: Skipped 1 previous similar message [ 3593.828538] Lustre: server umount lustre-OST0000 complete [ 3599.329144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3599.339444] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3599.358284] Lustre: Skipped 1 previous similar message [ 3599.366296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3604.140903] Lustre: server umount lustre-MDT0000 complete [ 3604.970277] LustreError: 127776:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3604.985565] LustreError: 127776:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3607.518142] LustreError: 128416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788622163 with bad export cookie 17254660703071055172 [ 3607.529584] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3607.532248] LustreError: 128416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3607.887504] Lustre: server umount lustre-MDT0001 complete [ 3616.928934] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:29:30 (1788622170) [ 3623.590246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3624.065912] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3624.151320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3628.928656] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3629.539336] LustreError: 130830:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3634.675551] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3637.558167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3643.184343] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3650.879520] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3656.301908] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3663.511225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3664.049697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3664.061618] Lustre: Skipped 1 previous similar message [ 3670.554166] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3677.765839] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3685.245542] Lustre: *** cfs_fail_loc=159, val=0*** [ 3700.983132] Lustre: lustre-MDT0000: Client 57d4094d-5dec-4307-b972-1338f3fc244e (at 192.168.206.38@tcp) reconnecting [ 3702.975486] Lustre: *** cfs_fail_loc=159, val=0*** [ 3702.977298] Lustre: Skipped 5 previous similar messages [ 3719.410696] Lustre: lustre-MDT0000: Client 57d4094d-5dec-4307-b972-1338f3fc244e (at 192.168.206.38@tcp) reconnecting [ 3739.104764] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3739.108354] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3739.113418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3739.113427] Lustre: Skipped 1 previous similar message [ 3739.121607] Lustre: Skipped 2 previous similar messages [ 3742.776086] Lustre: server umount lustre-OST0000 complete [ 3749.345071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3749.366028] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3749.373458] Lustre: Skipped 1 previous similar message [ 3752.802860] Lustre: server umount lustre-MDT0000 complete [ 3754.464945] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3754.492200] LustreError: 130831:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3756.487407] LustreError: 130812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788622312 with bad export cookie 17254660703071056047 [ 3756.491174] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3756.506412] LustreError: 130812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3756.802642] Lustre: server umount lustre-MDT0001 complete [ 3767.305248] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 3770.877397] Key type lgssc unregistered [ 3771.183624] LNet: 134607:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3771.204580] LNetError: 134607:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3772.274258] LNet: Removed LNI 192.168.206.138@tcp [ 3773.240765] Key type .llcrypt unregistered [ 3773.242422] Key type ._llcrypt unregistered [ 3796.141435] Key type ._llcrypt registered [ 3796.143302] Key type .llcrypt registered [ 3796.246524] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 3797.207808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3797.286320] alg: No test for adler32 (adler32-zlib) [ 3798.371351] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 3798.619618] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 3800.312287] Key type lgssc registered [ 3801.412958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3809.043559] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3809.069319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3810.526113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3814.586281] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3820.008533] LustreError: 136017:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3820.038584] LustreError: 136017:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3822.982504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3823.384819] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3827.351324] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3834.666976] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3839.591640] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3845.569640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3845.926780] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3852.478311] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 3855.157506] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3859.636539] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3869.752425] Lustre: *** cfs_fail_loc=159, val=0*** [ 3886.312610] Lustre: lustre-MDT0000: Client b8f7433d-7b21-429b-849c-cde49b1fb925 (at 192.168.206.38@tcp) reconnecting [ 3888.667584] Lustre: *** cfs_fail_loc=159, val=0*** [ 3888.675866] Lustre: Skipped 1 previous similar message [ 3904.236780] Lustre: lustre-MDT0001: Client b8f7433d-7b21-429b-849c-cde49b1fb925 (at 192.168.206.38@tcp) reconnecting [ 3905.848847] Lustre: *** cfs_fail_loc=159, val=0*** [ 3905.851293] Lustre: Skipped 3 previous similar messages [ 3922.159367] Lustre: lustre-MDT0001: Client b8f7433d-7b21-429b-849c-cde49b1fb925 (at 192.168.206.38@tcp) reconnecting [ 3928.228808] Lustre: 139058:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3931.314052] Lustre: *** cfs_fail_loc=159, val=0*** [ 3931.317989] Lustre: Skipped 4 previous similar messages [ 3947.769420] Lustre: lustre-MDT0000: Client f2c5dd13-7900-4270-b0a5-9871f3d178ad (at 192.168.206.38@tcp) reconnecting [ 3949.423183] Lustre: *** cfs_fail_loc=159, val=0*** [ 3949.427802] Lustre: Skipped 14 previous similar messages [ 3965.164564] Lustre: lustre-MDT0000: Client f2c5dd13-7900-4270-b0a5-9871f3d178ad (at 192.168.206.38@tcp) reconnecting [ 3966.594334] Lustre: 139345:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3987.429088] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3987.434822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3987.448374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3988.451553] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3988.452961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3988.455557] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3991.920283] Lustre: server umount lustre-OST0000 complete [ 3997.664967] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3997.676148] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3997.688324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3997.691764] Lustre: Skipped 1 previous similar message [ 4002.139585] Lustre: server umount lustre-MDT0000 complete [ 4003.811341] LustreError: 138716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4003.832747] LustreError: 138716:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4006.040108] LustreError: 135998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788622562 with bad export cookie 11717968413085063380 [ 4006.052572] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4006.061473] LustreError: 135998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4006.361320] Lustre: server umount lustre-MDT0001 complete [ 4016.763193] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4019.134102] Key type lgssc unregistered [ 4019.355859] LNet: 140425:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4019.363076] LNetError: 140425:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4019.385276] LNet: Removed LNI 192.168.206.138@tcp [ 4020.145303] Key type .llcrypt unregistered [ 4020.151411] Key type ._llcrypt unregistered [ 4045.387878] Key type ._llcrypt registered [ 4045.390528] Key type .llcrypt registered [ 4045.518925] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4046.632097] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4046.658623] alg: No test for adler32 (adler32-zlib) [ 4047.935123] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4048.226642] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4049.975154] Key type lgssc registered [ 4051.244221] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4059.159986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4059.182115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4060.663531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4065.737201] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4070.884569] LustreError: 141835:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4070.911471] LustreError: 141835:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4074.389954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4074.608740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4078.839306] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4085.877414] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4091.089366] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4098.679955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4099.187234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4100.248794] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4105.337578] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4105.834885] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4115.153508] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4120.785728] Lustre: 144421:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4124.134780] Lustre: 144471:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4125.869903] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4127.144655] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4131.833066] Lustre: 144618:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4132.700793] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4133.777288] Lustre: 144717:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4141.546648] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4141.550640] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.573630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4141.580702] Lustre: Skipped 1 previous similar message [ 4144.697495] Lustre: server umount lustre-OST0000 complete [ 4151.264313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4151.267129] LustreError: Skipped 1 previous similar message [ 4151.271612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4151.276807] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4151.279470] Lustre: Skipped 1 previous similar message [ 4154.842523] Lustre: server umount lustre-MDT0000 complete [ 4156.899983] LustreError: 144483:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4156.915930] LustreError: 144483:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4158.717696] LustreError: 141817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788622714 with bad export cookie 12929708868951830199 [ 4158.722445] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4158.730956] LustreError: 141817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4159.170068] Lustre: server umount lustre-MDT0001 complete [ 4170.478616] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4172.824660] Key type lgssc unregistered [ 4173.088126] LNet: 145777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4173.094907] LNetError: 145777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4173.114213] LNet: Removed LNI 192.168.206.138@tcp [ 4173.833142] Key type .llcrypt unregistered [ 4173.834454] Key type ._llcrypt unregistered [ 4195.818688] Key type ._llcrypt registered [ 4195.822589] Key type .llcrypt registered [ 4195.911877] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4196.818315] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4196.882571] alg: No test for adler32 (adler32-zlib) [ 4197.912039] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4198.132101] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4199.775182] Key type lgssc registered [ 4200.734623] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4207.843964] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4207.871993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4209.276629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4213.277156] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4219.876607] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4219.904702] LustreError: 147188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4221.176195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4221.548805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4225.362231] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4233.269344] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4239.103908] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4246.718660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4247.130382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4253.852914] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4254.337220] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4261.889085] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4266.614258] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4271.731391] Lustre: *** cfs_fail_loc=159, val=0*** [ 4288.232839] Lustre: lustre-MDT0000: Client 2e2096b1-90d1-4a7a-b49d-0071abe0eb13 (at 192.168.206.38@tcp) reconnecting [ 4305.892059] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4305.897633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4305.906491] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4307.936969] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4307.942147] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4307.964511] Lustre: Skipped 1 previous similar message [ 4311.556521] Lustre: server umount lustre-OST0000 complete [ 4318.177835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4318.200245] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4318.212409] Lustre: Skipped 1 previous similar message [ 4321.565852] Lustre: server umount lustre-MDT0000 complete [ 4323.296761] LustreError: 147919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4323.310133] LustreError: 147919:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4324.313392] LustreError: 147168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788622880 with bad export cookie 17674739859002314706 [ 4324.314929] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4324.327578] LustreError: 147168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4324.577318] Lustre: server umount lustre-MDT0001 complete [ 4334.601680] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4337.535879] Key type lgssc unregistered [ 4337.964719] LNet: 150855:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4337.971885] LNetError: 150855:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4338.002659] LNet: Removed LNI 192.168.206.138@tcp [ 4339.026407] Key type .llcrypt unregistered [ 4339.029562] Key type ._llcrypt unregistered [ 4363.942490] Key type ._llcrypt registered [ 4363.944878] Key type .llcrypt registered [ 4364.046320] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4365.037915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4365.087437] alg: No test for adler32 (adler32-zlib) [ 4366.175664] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4366.395750] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4368.031107] Key type lgssc registered [ 4369.177923] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4375.946180] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4375.970633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4377.343894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4381.283462] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4387.811354] LustreError: 152269:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4387.826808] LustreError: 152269:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4390.449481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4390.910333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4395.199591] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4403.075468] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4408.168381] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4414.973696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4415.449317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4420.601497] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4421.390606] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4422.640068] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4429.408650] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4448.445146] Lustre: lustre-MDT0001: evicting b97efed7-55c8-45ff-87bf-f112d7844bcc (at 192.168.206.38@tcp) by administrative request [ 4448.453275] Lustre: Skipped 1 previous similar message [ 4476.899441] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4476.908760] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4476.919636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4478.953154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4478.954719] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4478.958333] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4478.966082] Lustre: Skipped 1 previous similar message [ 4482.913676] Lustre: server umount lustre-OST0000 complete [ 4488.160450] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4488.173427] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4488.196701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4493.270441] Lustre: server umount lustre-MDT0000 complete [ 4494.309613] LustreError: 152270:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4494.337408] LustreError: 152270:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4497.158591] LustreError: 152248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788623053 with bad export cookie 13387957394892934943 [ 4497.164599] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4497.169627] LustreError: 152248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4497.591551] Lustre: server umount lustre-MDT0001 complete [ 4507.866741] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4510.608993] Key type lgssc unregistered [ 4510.951950] LNet: 156334:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4510.957653] LNetError: 156334:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4511.973859] LNet: Removed LNI 192.168.206.138@tcp [ 4512.893417] Key type .llcrypt unregistered [ 4512.896037] Key type ._llcrypt unregistered [ 4588.485613] Key type ._llcrypt registered [ 4588.488031] Key type .llcrypt registered [ 4588.621535] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4589.856221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4589.965416] alg: No test for adler32 (adler32-zlib) [ 4591.326614] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4591.553582] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4593.247202] Key type lgssc registered [ 4594.500762] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4602.183339] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4602.205304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4603.539299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4608.242658] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4613.091403] LustreError: 158820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4613.121348] LustreError: 158820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4615.728561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4616.156918] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4620.166756] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4627.248496] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4632.107682] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4638.120291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4638.446364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4641.533819] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4644.797176] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4648.701515] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4653.172975] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4662.242616] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4662.251064] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4662.269787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4664.288762] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4664.296138] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4664.303104] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4664.338838] Lustre: Skipped 1 previous similar message [ 4668.330393] Lustre: server umount lustre-OST0000 complete [ 4672.481236] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4672.490768] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4672.511156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4674.532043] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4674.535418] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4678.510069] Lustre: server umount lustre-MDT0000 complete [ 4679.649711] LustreError: 158820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4679.677485] LustreError: 158820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4682.242799] LustreError: 158800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788623238 with bad export cookie 10106052006721389553 [ 4682.254187] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4682.260910] LustreError: 158800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4682.590438] Lustre: server umount lustre-MDT0001 complete [ 4693.319897] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4695.405786] Key type lgssc unregistered [ 4695.678801] LNet: 162359:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4695.689150] LNetError: 162359:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4695.701773] LNet: Removed LNI 192.168.206.138@tcp [ 4696.406288] Key type .llcrypt unregistered [ 4696.408370] Key type ._llcrypt unregistered [ 4716.254983] Key type ._llcrypt registered [ 4716.256948] Key type .llcrypt registered [ 4716.335583] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4716.822120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4716.860293] alg: No test for adler32 (adler32-zlib) [ 4717.826758] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4718.013261] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4719.695459] Key type lgssc registered [ 4720.854422] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4735.965288] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:48:09 (1788623289) [ 4753.677122] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 4763.764767] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4795.135318] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4795.152758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4796.447825] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4796.485691] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4796.605412] Lustre: lustre-MDT0000: new disk, initializing [ 4796.684630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4796.705496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4801.255172] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4811.055366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4811.151827] Lustre: 166595:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4811.212584] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4811.217875] Lustre: Skipped 1 previous similar message [ 4811.286367] Lustre: lustre-MDT0001: new disk, initializing [ 4811.371983] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4811.411039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4811.424512] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4815.645986] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4824.773677] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4829.975754] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4837.723959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4837.981361] Lustre: lustre-OST0000: new disk, initializing [ 4837.986511] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4837.992313] Lustre: 168567:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4838.089604] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4843.664637] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4847.690617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4847.698401] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4847.815182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4852.502965] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4863.457687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4863.461051] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4863.473096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4863.479677] Lustre: Skipped 1 previous similar message [ 4868.605214] Lustre: server umount lustre-OST0000 complete [ 4872.675342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4872.681853] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4872.696466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4872.703791] Lustre: Skipped 1 previous similar message [ 4873.697638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4873.700687] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4878.079210] Lustre: server umount lustre-MDT0000 complete [ 4878.817530] LustreError: 166602:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4878.829464] LustreError: 166602:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4881.671802] LustreError: 166588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788623437 with bad export cookie 17919779309710312029 [ 4881.678880] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4881.686619] LustreError: 166588:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4882.173356] Lustre: server umount lustre-MDT0001 complete [ 4891.881100] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 4894.347681] Key type lgssc unregistered [ 4894.642425] LNet: 170470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4894.654346] LNetError: 170470:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4894.669150] LNet: Removed LNI 192.168.206.138@tcp [ 4895.326584] Key type .llcrypt unregistered [ 4895.328948] Key type ._llcrypt unregistered [ 4915.217914] Key type ._llcrypt registered [ 4915.223089] Key type .llcrypt registered [ 4915.335675] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 4916.115072] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4916.158069] alg: No test for adler32 (adler32-zlib) [ 4917.134539] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 4917.330455] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 4918.983174] Key type lgssc registered [ 4919.918707] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4926.546836] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4926.579631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4928.083140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4932.179986] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4938.216256] LustreError: 171861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4938.236472] LustreError: 171861:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4940.508340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4940.764690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4946.176205] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4954.129273] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4960.137903] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4967.146202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4967.448320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4973.502805] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 4973.632681] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4981.660287] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4992.562629] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4992.590184] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4992.612648] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4992.618634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.631334] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4993.646107] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4993.677436] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4993.708515] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4993.720794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.743684] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4994.726392] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4994.765243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.795632] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4994.808055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.822822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4995.752420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.693197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.716133] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4997.720036] Lustre: Skipped 2 previous similar messages [ 4997.732617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4997.739628] Lustre: Skipped 2 previous similar messages [ 4997.742465] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4997.746686] Lustre: Skipped 2 previous similar messages [ 4997.749576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.768943] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4997.779070] Lustre: Skipped 2 previous similar messages [ 4998.564643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.504352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.302662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.265698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.265698] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5002.273084] Lustre: Skipped 4 previous similar messages [ 5002.290610] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5002.300950] Lustre: Skipped 4 previous similar messages [ 5002.309177] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5002.315574] Lustre: Skipped 4 previous similar messages [ 5002.319486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.339877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5002.352181] Lustre: Skipped 4 previous similar messages [ 5003.252961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.117906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.078443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.992970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.021299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.914530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.863738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.797084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.775412] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5010.784135] Lustre: Skipped 8 previous similar messages [ 5010.810303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5010.823522] Lustre: Skipped 8 previous similar messages [ 5010.827809] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5010.833114] Lustre: Skipped 8 previous similar messages [ 5010.840491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.852727] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5010.866786] Lustre: Skipped 8 previous similar messages [ 5012.008444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.994826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.015692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.040345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.076137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.939446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.947811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.781124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.694646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.762276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.860616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.680764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.692562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.531398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.549356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.439510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.359643] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5027.364456] Lustre: Skipped 16 previous similar messages [ 5027.379599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5027.390112] Lustre: Skipped 16 previous similar messages [ 5027.394899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5027.402036] Lustre: Skipped 16 previous similar messages [ 5027.405811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.421553] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5027.432494] Lustre: Skipped 16 previous similar messages [ 5028.357231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.174234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.318297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.372458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.198834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.037509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.956303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.801579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.708904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.510396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.481891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.246717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.079506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.066921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.215982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.325356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.201465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.172827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.021357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.007161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.861710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.954204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.887548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.988865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.746434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.641804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.453467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.351713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.276283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.095409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.072428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.921602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.916550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.920595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.697402] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5059.701692] Lustre: Skipped 34 previous similar messages [ 5059.714190] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5059.721404] Lustre: Skipped 34 previous similar messages [ 5059.726724] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5059.736536] Lustre: Skipped 34 previous similar messages [ 5059.741808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.751290] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5059.756732] Lustre: Skipped 34 previous similar messages [ 5060.614342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.290173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.225980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.133540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.201479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.349793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.310085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.332651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.186081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.185786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.161702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.151200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.118952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.020877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.814534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.784917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.665818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.523416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.395893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.288125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.089795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.039572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.790116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.706440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.558969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.589271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.585625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.535715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.554691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.639405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.467212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.490672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.341775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.239052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.225651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.082914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.989114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.031542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.808371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.889505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.730548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.547901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.421886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.476843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.310200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.208804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.090590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.812690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.570769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.422324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.161348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.009127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.758548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.596100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.387488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.242289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.003588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.034297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.887199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.811839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.677443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.762444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.841556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.769671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.784824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.685148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.640187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.467152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.364300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.159169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.072114] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5124.076583] Lustre: Skipped 70 previous similar messages [ 5124.085895] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.101452] Lustre: Skipped 70 previous similar messages [ 5124.105548] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5124.109855] Lustre: Skipped 70 previous similar messages [ 5124.113858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.126842] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5124.133870] Lustre: Skipped 70 previous similar messages [ 5124.928513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.785941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.551826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.597256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.509709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.421821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.453971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.589544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.671584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.584651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.647198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.733252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.617314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.752915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.760525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.779801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.787304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.757051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.658269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.572734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.392118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.189399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.050612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.902843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.750797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.646661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.443669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.387186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.097791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.076881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.111121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.009838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.921692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.671340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.558720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.374535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.305658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.288948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.486238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.265889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.216571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.168511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.057213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.016984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.013338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.933459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.892870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.924545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.948573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.873122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.850845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.715717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.474976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.298923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.265628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.058364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.031417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.997385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.870861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.813925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.730785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.658780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.415088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.238274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.006370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.887896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.659188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.393898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.365936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.395304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.277317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.418782] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5190.443926] LustreError: 171881:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5190.477594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.785722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.043750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.162083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.264210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.532940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.665460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.701510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.754823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.769312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.604326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.532416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.526614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.462077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.501884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.484706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.610248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.305300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.273480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.097570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.002581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.811902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.954339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.749707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.624702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.574576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.453988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.404584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.391041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.560921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.591872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.477446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.384226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.575944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.891511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.302638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.647536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.628653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.822423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.830771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.043318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.166109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.237026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.506788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.628787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.436541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.608471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.550216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.983680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.986831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.870054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.969567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.139771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.539480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.920380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.149264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.468761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.474406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.627810] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5252.636106] Lustre: Skipped 129 previous similar messages [ 5252.658572] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5252.674120] Lustre: Skipped 129 previous similar messages [ 5252.679899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5252.689323] Lustre: Skipped 129 previous similar messages [ 5252.698214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.715239] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5252.736055] Lustre: Skipped 129 previous similar messages [ 5253.771072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.849443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.636843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.562278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.556981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.447169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.440835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.410867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.262730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.364151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.342900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.423298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.410897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.513709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.493535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.658421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.764914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.928326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.860578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.914200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.776234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.731955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.590876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.585551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.481050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.708527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.790651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.842363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.806913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.677550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.331292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.207426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.992650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.952764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.899830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.079220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.848090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.738690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.690384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.838315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.956621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.977825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.002718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.074372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.313760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.378561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.346980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.246413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.158969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.322505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.346372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.438588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.359910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.218801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.153647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.240509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.203457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.278061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.199655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.286186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.258091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.319279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.104339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.190526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.003758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.131757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.958962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.163943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.228039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.154441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.251975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.355433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.505922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.508428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.387415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.468483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.649838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.544661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.602593] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5331.614444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.614909] LustreError: 171881:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5332.806709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.965545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.876864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.816074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.709905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.693653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.559416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.557294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.583641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.410984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.368255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.339685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.286781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.290023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.247954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.240422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.279833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.163374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.003881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.912702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.762590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.644847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.597056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.739527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.784801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.851678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.789868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.734969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.734789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.736589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.509071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.259372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.023601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.824918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.525161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.463714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.383237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.265873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.010388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.775800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.609330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.567252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.333457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.086313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.879168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.665299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.438837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.175276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.900829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.675362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.511393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.344376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.037143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.770310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.538409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.390665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.153402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.080571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.046552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.931260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.654512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.465270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.308673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.228134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.064639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.774656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.593435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.401987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.292806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.095562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.949978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.697688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.527782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.518656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.444585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.364148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.329184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.266324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.356569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.104194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.014950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.850390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.716767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.488707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.185440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.964454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.869560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.694461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.466082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.145316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.865056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.508217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.204471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.910293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.762595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.450935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.337637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.178499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.084140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.864538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.640070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.457701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.250146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.991121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.776325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.461669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.155668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.045161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.148932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.907471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.872847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.733388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.590884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.427403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.329535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.215062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.155671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.920664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.808882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.660277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.700095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.776889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.949824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.085456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.929765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.292069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.246511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.350460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.328270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.427398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.413715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.380470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.233729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.136729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.960586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.778521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.605209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.466469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.291127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.295419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.391243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.241560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.079809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.897693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.829248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.798774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.781829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.723191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.983279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.969215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.967381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.933567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.796048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.632052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.714252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.752787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.748659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.752695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.767329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.718804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.616740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.501787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.489755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.486874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.612284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.434745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.296780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.218580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.216603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.186462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.973563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.836440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.774992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.747756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.675274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.574955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.560256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.439705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.501411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.412935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.323231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.208477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.351871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.250707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.940646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.604183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.332273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.097846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.971845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.844116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.829656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.582436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.480612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.219588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.971192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.879504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.921482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.877755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.958131] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5508.962888] Lustre: Skipped 277 previous similar messages [ 5508.981935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5508.993091] Lustre: Skipped 277 previous similar messages [ 5508.995889] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5508.998472] Lustre: Skipped 277 previous similar messages [ 5509.001054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.022261] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5509.030336] Lustre: Skipped 277 previous similar messages [ 5509.776305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.546231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.306762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.096339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.005340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.946371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.763494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.797302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.622462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.603722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.485208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.261721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.095769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.976425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.719232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.654198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.473399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.256301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.050315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.962099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.876485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.710182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.598045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.427486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.282755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.107130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.927534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.686460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.505401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.417854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.228745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.139697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.929651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.774273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.640347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.461255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.310241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.289328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.249481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.186384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.042274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.788730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.613761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.554232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.391887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.241473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.106385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.098973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.190628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.491647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.527954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.611224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.542389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.465375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.274372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.231357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.163687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.008566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.864606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.827876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.687446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.534412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.535633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.593443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.707323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.628355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.498269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.701863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.740578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.865700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.901100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.746308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.676952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.639286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.662197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.836900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.884405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.976812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.891136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.981605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.935856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.797613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.513679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.428576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.108903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.869499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.621332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.452636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.198071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.008559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.759229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.557147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.213912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.031970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.882317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.951697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.920194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.919976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.868408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.994557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.017846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.994491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.819779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.802945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.641055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.436538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.208380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.108917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.931724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.772392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.623042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.454181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.339492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.269344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.109982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.939281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.801570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.591959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.611542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.659789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.758462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.717054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.578365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.373662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.308861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.077728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.838831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.503378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.342961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.233924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.064081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.877496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.760766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.656926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.894745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.082547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.196294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.275798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.151669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.102555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.006821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.805321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.749551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.640588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.517866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.440638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.299614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.200187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.079083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.915806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.733651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.549476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.286710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.250452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.982459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.785756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.646992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.440922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.281310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.053135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.779491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.589259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.294427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.008838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.866282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.739673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.633044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.382289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.214427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.116633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.937984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.746842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.679304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.463082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.164453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.834764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.587797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.388655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.100957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.956517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.589448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.220973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.048697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.955265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.785411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.604274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.445587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.357331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.241615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.138684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.985399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.672633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.485919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.250862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.056702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.841884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.542865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.468432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.336847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.325198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.201526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.172716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.255598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.277883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.180198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.151432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.066655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.050843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.131859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.167844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.242465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.295087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.311949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.335862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.247817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.028790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.952921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.803443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.628093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.523385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.277379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.080405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.999890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.910397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.859509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.690801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.599722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.456293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.422863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.360610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.193513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.988025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.913196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.719512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.574535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.557539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.502378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.368315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.158649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.149889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.063489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.923057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.747637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.572600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.378008] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.264709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.190606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.177230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.915949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.732070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.612304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.510782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.340654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.264258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.982303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.807957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.543369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.374511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.226633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.006299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.871621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.688887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.482984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.343697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.184504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.144475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.049756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.051761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.989480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.033992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.815258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.866592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.659950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.638164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.555584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.380269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.196463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.977973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.718111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.461533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.137466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.806859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.420915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.197593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.254836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.210873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.450725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.246695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.152890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.993602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.900886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.712945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.870096] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5766.889094] LustreError: 171881:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5766.891982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.071956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.180779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.219070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.222894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.152920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.132820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.053923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.000699] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5775.007287] LustreError: 171881:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5775.010987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.866517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.759202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.851473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.814525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.848269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.617655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.475575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.291794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.446465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.472873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.520063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.122701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.780898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.543211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.482452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.267040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.201881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.987218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.020762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.147519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.247877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.329388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.446291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.249660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.938215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.924032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.978904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.945669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.850222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.693792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.687739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.923962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.890945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.813062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.755471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.691572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.536250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.502583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.367688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.308898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.208098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.255276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.247246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.401695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.667313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.497934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.523586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.444907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.471113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.351929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.417027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.431803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.265747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.247581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.218979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.109769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.136855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.049628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.242541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.169150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.082568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.158167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.086063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.977287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.690453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.620811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.644943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.656655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.751460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.676269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.559866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.372360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.344740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.215820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.272949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.343372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.192377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.096979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.993316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.108105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.032178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.864986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.780752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.739633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.520724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.417348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.354040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.263972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.214509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.133803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.893194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.635111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.413950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.231072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.124483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.956204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.775613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.573346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.475992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.373656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.188321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.989038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.850638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.772253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.571447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.402177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.297555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.192219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.051222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.913713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.733629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.554052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.291267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.003556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.887519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.719338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.487417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.200320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.019670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.978795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.829220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.688315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.482696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.468258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.337927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.165661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.841991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.692229] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5891.701663] LustreError: 171881:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5891.704738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.418185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.199734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.906113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.592529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.334662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.140436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.778430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.555135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.305651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.055698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.931858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.704660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.465646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.233173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.140481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.064072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.830069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.520229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.270529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.048798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.265174] LustreError: 173685:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6278.277640] LustreError: 171881:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6924.272296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6924.276438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6924.285428] Lustre: Skipped 450 previous similar messages [ 6924.292610] Lustre: Skipped 1 previous similar message [ 6929.395545] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6929.409098] Lustre: Skipped 1 previous similar message [ 6930.028637] Lustre: server umount lustre-OST0000 complete [ 6934.509304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6934.514687] Lustre: Skipped 1 previous similar message [ 6939.620154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6939.628659] Lustre: Skipped 1 previous similar message [ 6944.739146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6944.750285] Lustre: Skipped 1 previous similar message [ 6948.320860] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6948.687164] Lustre: server umount lustre-MDT0000 complete [ 6949.860787] LustreError: 172821:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6949.874436] LustreError: 172821:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6952.823432] LustreError: 172596:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788625508 with bad export cookie 713260394333032127 [ 6952.833092] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6952.836922] LustreError: 172596:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6953.128656] Lustre: server umount lustre-MDT0001 complete [ 6963.998134] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 6967.180936] Key type lgssc unregistered [ 6967.557655] LNet: 227119:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6967.573947] LNetError: 227119:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6967.599055] LNet: Removed LNI 192.168.206.138@tcp [ 6968.437420] Key type .llcrypt unregistered [ 6968.439944] Key type ._llcrypt unregistered [ 6994.609823] Key type ._llcrypt registered [ 6994.612740] Key type .llcrypt registered [ 6994.748192] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 6995.788763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6996.063497] alg: No test for adler32 (adler32-zlib) [ 6997.188367] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 6997.536700] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 6999.232080] Key type lgssc registered [ 7000.242763] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7007.525662] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7007.558623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7009.045462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7013.516692] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7018.485139] LustreError: 228534:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7018.520800] LustreError: 228534:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7023.134839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7023.655611] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7028.452774] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7036.002946] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7041.772614] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7048.054728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7048.176648] LustreError: 230354:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7048.358009] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7053.487805] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7053.988389] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7060.967080] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7065.913651] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7068.094850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7068.106343] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7068.113398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.122755] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7068.126610] LustreError: 228553:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7073.002702] Lustre: lustre-OST0000: Not available for connect from 192.168.206.38@tcp (stopping) [ 7074.273777] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7074.299416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7074.306886] Lustre: Skipped 1 previous similar message [ 7076.785603] Lustre: server umount lustre-OST0000 complete [ 7088.183125] LustreError: 228553:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7111.596746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7111.947288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7114.026950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.033939] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.034636] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7114.050186] Lustre: Skipped 1 previous similar message [ 7117.935839] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7125.312636] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7127.104460] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7132.568730] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff902e89465800.ost_server_uuid 50 [ 7134.325792] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff902e89465800.ost_server_uuid in FULL state after 0 sec [ 7139.812659] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7139.814108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7139.828558] Lustre: Skipped 1 previous similar message [ 7139.831269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7139.843843] LustreError: Skipped 1 previous similar message [ 7144.434872] Lustre: server umount lustre-OST0000 complete [ 7150.048926] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7150.067499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7150.075867] Lustre: Skipped 2 previous similar messages [ 7150.087439] Lustre: Skipped 2 previous similar messages [ 7155.170595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7155.177655] Lustre: Skipped 1 previous similar message [ 7162.335180] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7162.597496] Lustre: server umount lustre-MDT0000 complete [ 7164.392822] LustreError: 229261:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7164.418238] LustreError: 229261:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7166.373026] LustreError: 228514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788625722 with bad export cookie 11539210022014171839 [ 7166.377773] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7166.385378] LustreError: 228514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7166.733585] Lustre: server umount lustre-MDT0001 complete [ 7177.353831] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 7179.895815] Key type lgssc unregistered [ 7180.177709] LNet: 233742:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7180.188965] LNetError: 233742:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7180.210394] LNet: Removed LNI 192.168.206.138@tcp [ 7180.857402] Key type .llcrypt unregistered [ 7180.861879] Key type ._llcrypt unregistered [ 7211.922441] Key type ._llcrypt registered [ 7211.927856] Key type .llcrypt registered [ 7212.064336] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 7212.936067] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7212.975786] alg: No test for adler32 (adler32-zlib) [ 7214.084438] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 7214.368108] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 7216.127448] Key type lgssc registered [ 7218.041764] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7227.332521] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7227.353252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7228.786880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7233.956316] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7238.118508] LustreError: 235470:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7238.144169] LustreError: 235470:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7241.224837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7241.580951] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7244.838601] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7247.816206] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7253.374854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7253.615422] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7254.637251] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7259.172289] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7267.548844] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7267.744944] Lustre: lustre-OST0001: new disk, initializing [ 7267.752346] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7267.760272] Lustre: 237777:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7267.873108] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7274.181319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7274.191250] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7274.248833] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7274.387926] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7284.502787] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7293.290869] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7308.260927] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7308.280225] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7308.297262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7310.305585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7310.307892] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7310.314924] Lustre: Skipped 2 previous similar messages [ 7310.319701] Lustre: Skipped 2 previous similar messages [ 7313.428670] Lustre: server umount lustre-MDT0000 complete [ 7315.434965] LustreError: 235465:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7315.451337] LustreError: 235465:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7316.912991] LustreError: 238545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788625872 with bad export cookie 706999479993091502 [ 7316.914917] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7316.933987] LustreError: 238545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7317.503292] Lustre: server umount lustre-MDT0001 complete [ 7331.291579] Lustre: server umount lustre-OST0000 complete [ 7345.431632] Lustre: server umount lustre-OST0001 complete [ 7367.351576] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 7375.485973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7375.857185] LustreError: 240883:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7375.944268] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7379.819247] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7380.963293] LustreError: 240884:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7385.064809] LustreError: 240883:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7387.828400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7388.154497] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7392.424328] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7395.335482] Lustre: 242025:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7401.407356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7407.607352] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7413.989109] LustreError: 242379:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7416.202740] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7416.452279] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7416.459523] Lustre: Skipped 1 previous similar message [ 7420.552084] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7420.552335] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7422.606001] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7430.346859] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7433.997177] Lustre: 243895:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7446.497783] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7446.502465] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7446.537016] Lustre: Skipped 3 previous similar messages [ 7446.561056] Lustre: Skipped 4 previous similar messages [ 7451.625674] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7451.632713] Lustre: Skipped 3 previous similar messages [ 7456.738175] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7456.746069] Lustre: Skipped 3 previous similar messages [ 7459.295146] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7459.475881] Lustre: server umount mylustre-MDT0000 complete [ 7461.866844] LustreError: 240878:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7461.878272] LustreError: 240878:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7462.549115] LustreError: 243898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626018 with bad export cookie 706999479993094309 [ 7462.557793] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7462.560838] LustreError: 243898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7462.891911] Lustre: server umount mylustre-MDT0001 complete [ 7476.286948] Lustre: server umount mylustre-OST0000 complete [ 7490.128748] Lustre: server umount mylustre-OST0001 complete [ 7507.446886] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 7516.843869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7517.324763] LustreError: 246206:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7517.394808] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7521.182119] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7528.437762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7533.129445] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7535.767826] Lustre: 247348:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7541.639872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7542.976220] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7548.182373] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7556.587439] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7562.236142] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7563.080277] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7571.118921] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7574.955695] Lustre: 249219:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7582.695243] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7582.698833] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7582.712317] Lustre: Skipped 3 previous similar messages [ 7582.729884] Lustre: Skipped 3 previous similar messages [ 7583.983481] Lustre: server umount tfs-MDT0000 complete [ 7587.515260] LustreError: 246181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626143 with bad export cookie 706999479993096199 [ 7587.525637] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7587.542178] LustreError: 246181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7587.810585] LustreError: 246201:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7587.812368] Lustre: tfs-MDT0001-lwp-OST0000: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7587.822274] LustreError: 246201:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7587.850528] Lustre: Skipped 1 previous similar message [ 7587.950258] Lustre: server umount tfs-MDT0001 complete [ 7593.489905] Lustre: server umount tfs-OST0000 complete [ 7596.728217] Lustre: server umount tfs-OST0001 complete [ 7614.010801] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 7622.257976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7622.672170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7622.677132] Lustre: Skipped 3 previous similar messages [ 7626.584791] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7634.225636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7638.531851] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7641.108466] Lustre: 252609:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7647.622301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7653.891547] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7661.567788] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7666.903807] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7668.217597] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7668.224074] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7673.246818] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7676.252746] Lustre: 254477:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7681.858046] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:37:15 (1788626235) [ 7698.915762] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7698.920912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7698.931789] Lustre: Skipped 3 previous similar messages [ 7698.939137] Lustre: Skipped 2 previous similar messages [ 7702.789567] Lustre: server umount lustre-MDT0000 complete [ 7705.658394] LustreError: 251440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626261 with bad export cookie 706999479993097865 [ 7705.659427] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7705.669551] LustreError: 251440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7712.109674] Lustre: server umount lustre-MDT0001 complete [ 7722.137764] Lustre: server umount lustre-OST0000 complete [ 7740.385488] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7740.629394] Lustre: server umount lustre-OST0001 complete [ 7746.542932] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 7755.714175] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 7791.029504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7791.208978] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7791.242671] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7791.315316] Lustre: lustre-MDT0000: new disk, initializing [ 7791.390885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7791.397394] Lustre: Skipped 3 previous similar messages [ 7791.418643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7795.848087] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7805.139989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7805.225621] Lustre: 257660:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7805.296967] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7805.301102] Lustre: Skipped 1 previous similar message [ 7805.356977] Lustre: lustre-MDT0001: new disk, initializing [ 7805.456170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7805.470049] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7808.905824] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7816.699788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7816.882804] Lustre: lustre-OST0000: new disk, initializing [ 7816.885705] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7816.890874] Lustre: 259243:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7818.095858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7818.106285] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7818.171827] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7822.322852] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7832.168782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7832.391422] Lustre: lustre-OST0001: new disk, initializing [ 7832.398880] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7832.407836] Lustre: 260116:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7833.962058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7833.972455] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7834.033511] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7839.071516] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 7851.769836] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:40:05 (1788626405) [ 7867.629278] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:40:21 (1788626421) [ 7884.768609] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7884.769874] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7884.776365] LustreError: Skipped 1 previous similar message [ 7884.778175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7884.793179] Lustre: Skipped 3 previous similar messages [ 7884.810572] Lustre: Skipped 5 previous similar messages [ 7888.019385] Lustre: server umount lustre-OST0000 complete [ 7890.409346] LustreError: 259227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7890.434637] LustreError: 259227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 16 previous similar messages [ 7892.450177] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7898.027395] Lustre: server umount lustre-MDT0000 complete [ 7901.418350] LustreError: 260115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626457 with bad export cookie 706999479993099419 [ 7901.421736] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7901.428168] LustreError: 260115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7907.823663] Lustre: server umount lustre-MDT0001 complete [ 7918.650243] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 7992.387679] Lustre: server umount lustre-OST0001 complete [ 7999.876566] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 8014.216132] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8044.822940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8045.040218] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8045.060919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8045.134205] Lustre: lustre-MDT0000: new disk, initializing [ 8045.221397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8049.131429] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8058.176974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8058.310616] Lustre: 264965:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8058.499854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8058.505792] Lustre: Skipped 4 previous similar messages [ 8058.557733] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8062.243107] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8070.685601] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8075.920561] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8082.208478] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8082.472475] Lustre: lustre-OST0000: new disk, initializing [ 8082.474993] Lustre: Skipped 1 previous similar message [ 8082.480266] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8082.484707] Lustre: Skipped 2 previous similar messages [ 8082.494161] Lustre: 266931:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8084.418321] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8084.429215] Lustre: Skipped 1 previous similar message [ 8084.442620] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8084.559509] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8088.767926] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8096.802595] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8104.935276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8104.938934] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8104.955650] Lustre: Skipped 4 previous similar messages [ 8104.960176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8104.960195] Lustre: Skipped 8 previous similar messages [ 8109.206289] Lustre: server umount lustre-OST0000 complete [ 8115.168583] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8115.181865] LustreError: Skipped 1 previous similar message [ 8119.522086] Lustre: server umount lustre-MDT0000 complete [ 8122.998375] LustreError: 264958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626678 with bad export cookie 706999479993100882 [ 8123.005016] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8123.010510] LustreError: 264958:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8123.541302] Lustre: server umount lustre-MDT0001 complete [ 8137.220160] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 8140.652845] Key type lgssc unregistered [ 8140.912646] LNet: 268692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8140.922890] LNetError: 268692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8140.948746] LNet: Removed LNI 192.168.206.138@tcp [ 8142.241180] Key type .llcrypt unregistered [ 8142.249899] Key type ._llcrypt unregistered [ 8171.714783] Key type ._llcrypt registered [ 8171.719623] Key type .llcrypt registered [ 8171.822467] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8172.793071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8172.851549] alg: No test for adler32 (adler32-zlib) [ 8173.953407] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 8174.229723] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 8175.895672] Key type lgssc registered [ 8177.059744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8184.846274] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8184.874335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8186.384426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8190.737237] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8196.579290] LustreError: 270152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8196.609367] LustreError: 270152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8198.901022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8199.215429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8204.096958] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8214.022660] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8222.771188] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8229.551962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8230.036481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8231.104626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8237.660377] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8245.882244] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8247.016799] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8249.830650] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8249.836657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8249.848782] Lustre: Skipped 1 previous similar message [ 8249.860083] Lustre: Skipped 1 previous similar message [ 8251.310989] Lustre: server umount lustre-OST0000 complete [ 8254.946222] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8254.958026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8254.963608] Lustre: Skipped 1 previous similar message [ 8260.064274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8260.074902] Lustre: Skipped 1 previous similar message [ 8265.184441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8265.190177] Lustre: Skipped 1 previous similar message [ 8269.280980] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8269.287640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8269.301944] Lustre: Skipped 1 previous similar message [ 8269.486753] Lustre: server umount lustre-MDT0000 complete [ 8272.835568] LustreError: 270132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788626828 with bad export cookie 3132179436619034220 [ 8272.837840] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8272.843623] LustreError: 270132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8273.238157] Lustre: server umount lustre-MDT0001 complete [ 8282.758460] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 8284.941032] Key type lgssc unregistered [ 8285.222291] LNet: 273736:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8285.233202] LNetError: 273736:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8285.258917] LNet: Removed LNI 192.168.206.138@tcp [ 8285.951216] Key type .llcrypt unregistered [ 8285.954090] Key type ._llcrypt unregistered [ 8301.852607] Key type ._llcrypt registered [ 8301.854628] Key type .llcrypt registered [ 8301.964321] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8303.075297] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8303.196566] alg: No test for adler32 (adler32-zlib) [ 8304.371163] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 8304.609907] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 8306.255968] Key type lgssc registered [ 8307.293314] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8315.607730] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8315.627331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8331.167713] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8336.287924] LustreError: 274925:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.138@tcp: failed processing log, type 4: rc = -110 [ 8362.911939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8369.350721] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8375.680216] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8377.610441] Lustre: server umount lustre-OST0000 complete [ 8392.820854] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 8395.658781] Key type lgssc unregistered [ 8395.976681] LNet: 276556:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8395.984581] LNetError: 276556:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8396.001162] LNet: Removed LNI 192.168.206.138@tcp [ 8396.810220] Key type .llcrypt unregistered [ 8396.821124] Key type ._llcrypt unregistered [ 8433.319398] Key type ._llcrypt registered [ 8433.325546] Key type .llcrypt registered [ 8433.456650] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8434.498925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8434.529263] alg: No test for adler32 (adler32-zlib) [ 8435.543079] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 8435.850592] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 8437.511469] Key type lgssc registered [ 8438.434703] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8453.577716] loop0: detected capacity change from 0 to 204800000 [ 8453.897861] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8453.928588] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8454.697857] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8455.832678] loop0: detected capacity change from 0 to 204800000 [ 8456.114631] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8456.133080] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8456.571807] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.640722] loop0: detected capacity change from 0 to 204800000 [ 8457.811534] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.834645] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.935768] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8459.166668] loop0: detected capacity change from 0 to 204800000 [ 8459.417611] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8459.432479] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8459.520919] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8460.490794] loop0: detected capacity change from 0 to 204800000 [ 8460.512646] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8464.344814] loop0: detected capacity change from 0 to 204800000 [ 8464.371657] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8468.481333] loop0: detected capacity change from 0 to 204800000 [ 8468.520781] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8472.043980] loop0: detected capacity change from 0 to 204800000 [ 8472.068785] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8475.649244] loop0: detected capacity change from 0 to 204800000 [ 8475.716142] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8475.758154] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8475.820122] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8475.833967] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8475.845870] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8475.857664] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8475.865179] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8475.875544] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8475.885851] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8475.894809] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8475.903615] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8475.918328] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8475.928989] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8475.940259] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8475.948304] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8475.958658] Lustre: 279967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8481.033419] Lustre: server umount MGS complete [ 8481.907423] loop0: detected capacity change from 0 to 204800000 [ 8481.983670] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8482.314620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8482.355462] Lustre: lustre-MDT0000: Aborting client recovery [ 8483.104305] loop1: detected capacity change from 0 to 204800000 [ 8483.204103] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8483.313852] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8483.325327] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8483.341121] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8483.364990] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8483.385908] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8483.400252] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8483.414520] Lustre: 280400:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8483.432507] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8483.587458] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8483.626304] Lustre: lustre-MDT0001: Aborting client recovery [ 8484.382849] loop2: detected capacity change from 0 to 204800000 [ 8484.467138] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8484.578521] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8484.588961] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8484.596745] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8484.605432] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8484.612585] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8484.619789] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8484.626754] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8484.753854] Lustre: Found index 0 for lustre-OST0000, updating log [ 8484.779701] Lustre: 280497:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8484.844680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8484.858023] Lustre: lustre-OST0000: Aborting recovery [ 8485.552302] loop3: detected capacity change from 0 to 204800000 [ 8485.639903] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8485.761979] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8485.789791] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8485.799933] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8485.812435] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8485.822643] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8485.831469] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8485.842451] Lustre: 280563:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8485.859215] Lustre: Found index 1 for lustre-OST0001, updating log [ 8485.889602] Lustre: 280570:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8485.947306] Lustre: lustre-OST0001: Aborting recovery [ 8490.999331] LustreError: 280344:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8492.011663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8492.011822] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8492.049309] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8492.072102] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8493.053482] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8493.056248] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8493.977418] LustreError: 280311:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 549us: rc = -2 [ 8494.538441] LustreError: 280311:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3880us: rc = -2 [ 8494.551972] LustreError: 280311:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8508.384786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8508.390574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8508.407313] Lustre: Skipped 2 previous similar messages [ 8508.424900] Lustre: Skipped 3 previous similar messages [ 8513.505689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8518.624035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8518.629588] Lustre: Skipped 6 previous similar messages [ 8521.183234] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8521.426091] Lustre: server umount lustre-MDT0000 complete [ 8522.216434] LustreError: 280295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788627078 with bad export cookie 15120642127548669978 [ 8522.223611] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8522.225901] LustreError: 280295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8523.743732] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8523.745901] LustreError: 280309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8523.746069] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8523.763115] Lustre: Skipped 2 previous similar messages [ 8523.779972] LustreError: 280309:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8527.775793] LustreError: 280507:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8527.775803] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8527.775808] Lustre: Skipped 1 previous similar message [ 8527.792143] LustreError: 280507:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8528.591514] Lustre: server umount lustre-MDT0001 complete [ 8535.401510] Lustre: server umount lustre-OST0000 complete [ 8550.367247] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8550.621528] Lustre: server umount lustre-OST0001 complete [ 8559.330986] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:51:52 (1788627112) [ 8585.504480] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 8594.082713] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8628.057226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8628.231253] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8628.265628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8628.366267] Lustre: lustre-MDT0000: new disk, initializing [ 8628.446574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8628.453544] Lustre: Skipped 1 previous similar message [ 8628.473406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8632.673834] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8641.835701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8641.923970] Lustre: 284639:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8641.985182] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8641.992686] Lustre: Skipped 1 previous similar message [ 8642.082720] Lustre: lustre-MDT0001: new disk, initializing [ 8642.234243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8642.283989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8642.302670] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8646.419396] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8655.359989] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8661.559433] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8668.577286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8668.874870] Lustre: lustre-OST0000: new disk, initializing [ 8668.880610] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8668.892430] Lustre: 286605:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8668.999123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8674.934531] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8678.991627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8679.000611] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8679.102808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8684.318026] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8689.061946] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8703.294529] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8708.949790] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8713.334560] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8766.433815] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8766.436102] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8766.457972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8766.462660] Lustre: Skipped 1 previous similar message [ 8776.159831] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8776.363184] Lustre: server umount lustre-OST0000 complete [ 8780.769821] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8780.777963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8780.792322] Lustre: Skipped 1 previous similar message [ 8786.788630] Lustre: server umount lustre-MDT0000 complete [ 8786.920838] LustreError: 284649:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8786.936747] LustreError: 284649:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8790.516335] LustreError: 284629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788627346 with bad export cookie 15120642127548672680 [ 8790.524494] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8790.527779] LustreError: 284629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8790.947618] Lustre: server umount lustre-MDT0001 complete [ 8799.793786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8804.282299] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8809.197319] Lustre: server umount MGS complete [ 8821.598975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8822.187422] LustreError: 290573:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8822.221768] LustreError: 290586:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8822.299663] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8826.750140] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8827.366229] LustreError: 290574:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8835.495474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8840.722387] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8849.253410] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8855.576566] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8861.384743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8861.628731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8861.636515] Lustre: Skipped 1 previous similar message [ 8866.444602] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 8872.609884] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8898.535638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8898.544548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8898.557593] Lustre: Skipped 1 previous similar message [ 8898.561818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8898.565558] Lustre: Skipped 5 previous similar messages [ 8900.589824] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8902.157794] Lustre: server umount lustre-OST0000 complete [ 8907.744142] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8907.753862] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8907.775271] Lustre: Skipped 1 previous similar message [ 8912.248754] Lustre: server umount lustre-MDT0000 complete [ 8915.888748] LustreError: 290554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788627471 with bad export cookie 15120642127548674507 [ 8915.896244] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8915.899506] LustreError: 290554:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8916.298977] Lustre: server umount lustre-MDT0001 complete [ 8927.076791] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 8930.083564] Key type lgssc unregistered [ 8930.366158] LNet: 294516:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8930.375102] LNetError: 294516:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8930.389373] LNet: Removed LNI 192.168.206.138@tcp [ 8931.274557] Key type .llcrypt unregistered [ 8931.277225] Key type ._llcrypt unregistered [ 8970.060577] Key type ._llcrypt registered [ 8970.063913] Key type .llcrypt registered [ 8970.208651] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 8987.154993] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 8988.698388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8988.734378] alg: No test for adler32 (adler32-zlib) [ 8989.828647] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 8990.135560] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 8991.778633] Key type lgssc registered [ 8992.894883] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9033.276494] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9033.315961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9034.721133] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9034.776510] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9034.903441] Lustre: lustre-MDT0000: new disk, initializing [ 9035.111482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9035.143366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9040.690837] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9050.294369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9050.412453] Lustre: 298471:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9050.487988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9050.492668] Lustre: Skipped 1 previous similar message [ 9050.622261] Lustre: lustre-MDT0001: new disk, initializing [ 9050.729547] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9050.781908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9050.796901] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9055.066683] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9065.553594] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9071.706510] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9078.999733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9079.444190] Lustre: lustre-OST0000: new disk, initializing [ 9079.454631] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9079.466743] Lustre: 300441:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9079.578580] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9086.386713] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9087.591305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9087.601711] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9087.739738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9096.899582] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9102.472731] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9113.769374] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9121.286151] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9129.979030] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9175.013856] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9175.015949] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9175.026432] LustreError: Skipped 1 previous similar message [ 9175.026560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9175.066467] Lustre: Skipped 1 previous similar message [ 9176.812512] Lustre: server umount lustre-OST0000 complete [ 9183.716990] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9183.723185] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9183.738611] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9183.744918] Lustre: Skipped 1 previous similar message [ 9185.250384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9185.250911] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9186.957589] Lustre: server umount lustre-MDT0000 complete [ 9190.674984] LustreError: 298462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788627746 with bad export cookie 11664665366930402173 [ 9190.681811] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9190.684882] LustreError: 298462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9191.057563] Lustre: server umount lustre-MDT0001 complete [ 9199.588246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9204.139115] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9208.870599] Lustre: server umount MGS complete [ 9220.598398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9221.113845] LustreError: 304370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9221.133335] LustreError: 304370:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9221.140604] LustreError: 304383:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9221.228598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9225.660964] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9226.723808] LustreError: 304371:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9231.844810] LustreError: 304370:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9235.204366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9235.601130] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9240.095650] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9246.569424] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9251.373387] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9257.916325] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9258.212940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9264.429239] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9272.429293] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9301.471935] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9301.476133] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9301.486883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9301.491271] Lustre: Skipped 1 previous similar message [ 9307.621722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9307.631464] Lustre: Skipped 3 previous similar messages [ 9313.247378] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9313.405593] Lustre: server umount lustre-OST0000 complete [ 9317.345620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9317.354922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9317.364513] Lustre: Skipped 1 previous similar message [ 9317.369029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9317.374817] Lustre: Skipped 2 previous similar messages [ 9331.167142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9331.395703] Lustre: server umount lustre-MDT0000 complete [ 9333.217822] LustreError: 304371:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9333.234798] LustreError: 304371:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9335.220124] LustreError: 304352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788627891 with bad export cookie 11664665366930404112 [ 9335.222111] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9335.231454] LustreError: 304352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9335.449209] Lustre: server umount lustre-MDT0001 complete [ 9344.398963] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [ 9346.947035] Key type lgssc unregistered [ 9347.267180] LNet: 308317:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9347.294800] LNetError: 308317:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9347.317690] LNet: Removed LNI 192.168.206.138@tcp [ 9348.431696] Key type .llcrypt unregistered [ 9348.438133] Key type ._llcrypt unregistered [ 9371.099170] Key type ._llcrypt registered [ 9371.100663] Key type .llcrypt registered [ 9371.212642] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [ 9384.214913] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 9385.109209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9385.220026] alg: No test for adler32 (adler32-zlib) [ 9386.323396] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [ 9386.593697] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [ 9388.255498] Key type lgssc registered [ 9389.485712] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9414.157685] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [ 9429.413768] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9429.442388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9430.734941] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9430.763501] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9430.834800] Lustre: lustre-MDT0000: new disk, initializing [ 9430.920652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9430.944598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9435.684985] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [ 9445.411524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9445.667658] Lustre: lustre-OST0000: new disk, initializing [ 9445.675275] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9445.682531] Lustre: Skipped 1 previous similar message [ 9445.688385] Lustre: 312341:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9445.779343] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9447.125455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9447.131812] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9447.161390] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9451.178345] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10317.793740] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10317.816214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10319.644811] Lustre: server umount lustre-MDT0000 complete [10322.991922] LustreError: 311528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788628878 with bad export cookie 7962643316408532636 [10322.997962] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10323.000504] LustreError: 311528:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10323.169305] Lustre: server umount lustre-OST0000 complete [10347.043749] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [10356.368413] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [10392.552469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10392.790255] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10392.829820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10392.950169] Lustre: lustre-MDT0000: new disk, initializing [10393.064068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10393.083881] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10397.118468] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10407.339788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10407.504372] Lustre: 316683:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10407.580941] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10407.587100] Lustre: Skipped 1 previous similar message [10407.659940] Lustre: lustre-MDT0001: new disk, initializing [10407.750128] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10407.804049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10407.812597] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10412.429446] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10422.179221] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10428.161813] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10434.599748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10434.981460] Lustre: lustre-OST0000: new disk, initializing [10434.990280] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10434.995784] Lustre: 318651:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10435.101041] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10440.846848] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10442.787711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10442.795412] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10442.902799] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10448.955839] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10452.854692] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10453.056240] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10456.607445] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10456.765974] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10458.597175] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10458.603442] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10458.615513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10463.201598] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10463.210508] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10463.230882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10464.429778] Lustre: server umount lustre-OST0000 complete [10468.832976] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10468.838732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10468.847377] Lustre: Skipped 1 previous similar message [10468.851104] Lustre: Skipped 1 previous similar message [10473.393490] Lustre: server umount lustre-MDT0000 complete [10476.528839] LustreError: 316675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629032 with bad export cookie 7962643316411473455 [10476.547580] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10476.813483] Lustre: server umount lustre-MDT0001 complete [10479.530861] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10480.707248] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:23:54 (1788629034) [10486.697949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10487.019656] LustreError: 320961:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10487.031023] LustreError: 320961:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10487.095976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10490.926881] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10492.387957] LustreError: 320962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10497.506603] LustreError: 320961:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10498.798248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10499.209084] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10502.971409] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10509.771388] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10514.333142] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10520.085059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10520.510602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10525.345392] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10531.894712] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10540.023815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10540.199073] Lustre: lustre-OST0001: new disk, initializing [10540.208518] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10540.212853] Lustre: 323799:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10540.315820] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10546.943505] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10546.972723] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10547.050770] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10548.070406] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10548.083578] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10548.460761] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10549.190337] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10549.211770] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10551.337728] LustreError: 324096:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10551.352530] LustreError: 324096:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10555.451437] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10555.471208] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10560.054851] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10560.370488] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10563.688957] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10563.698440] LustreError: 324090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10565.822237] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff902e87d45000.ost_server_uuid 50 [10567.178328] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff902e87d45000.ost_server_uuid in FULL state after 0 sec [10572.153500] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff902e87d45000.ost_server_uuid 50 [10573.683241] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff902e87d45000.ost_server_uuid in FULL state after 0 sec [10576.062565] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10588.136446] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10588.138528] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10588.147619] Lustre: Skipped 1 previous similar message [10588.153539] Lustre: Skipped 1 previous similar message [10594.085952] Lustre: server umount lustre-OST0001 complete [10595.520108] LustreError: 322785:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10599.401585] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10599.410578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10599.422191] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10599.429528] Lustre: Skipped 3 previous similar messages [10601.442697] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10603.489925] LustreError: 323459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10603.503835] LustreError: 323459:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10605.252588] Lustre: server umount lustre-OST0000 complete [10614.799809] Lustre: server umount lustre-MDT0000 complete [10618.242178] LustreError: 320941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629174 with bad export cookie 7962643316411474330 [10618.243637] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10618.251016] LustreError: 320941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10618.659166] Lustre: server umount lustre-MDT0001 complete [10627.590697] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [10629.966162] Key type lgssc unregistered [10630.165631] LNet: 326483:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10630.170136] LNetError: 326483:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10630.179704] LNet: Removed LNI 192.168.206.138@tcp [10630.909615] Key type .llcrypt unregistered [10630.916229] Key type ._llcrypt unregistered [10652.069181] Key type ._llcrypt registered [10652.073449] Key type .llcrypt registered [10652.189628] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [10653.158987] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10653.189528] alg: No test for adler32 (adler32-zlib) [10654.260400] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [10654.508500] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [10656.191137] Key type lgssc registered [10657.146932] Lustre: Echo OBD driver; http://www.lustre.org/ [10664.572238] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10664.590876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10666.015446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10670.350947] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10675.170469] LustreError: 327880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10675.188659] LustreError: 327880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10676.817744] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10680.292347] LustreError: 327879:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10683.165798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10683.493854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10687.631502] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10697.900106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10698.173786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10703.772797] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10708.464620] LustreError: 329558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10710.516744] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10710.534550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10710.540422] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10710.986971] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10715.617344] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10715.626702] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10717.008169] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10717.308056] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10722.816415] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10722.952270] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10730.070415] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10736.181064] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10736.356846] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10840.546706] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10840.556585] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10840.573302] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10840.578473] Lustre: Skipped 1 previous similar message [10840.594550] Lustre: Skipped 1 previous similar message [10845.667942] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10850.787454] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10850.791821] Lustre: Skipped 1 previous similar message [10851.298967] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10851.412930] Lustre: server umount lustre-OST0001 complete [10855.909625] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10855.929093] LustreError: 329560:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10856.930705] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10856.955175] LustreError: Skipped 1 previous similar message [10856.966739] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10856.986753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10858.978624] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10858.983359] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10861.027312] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10861.033234] Lustre: Skipped 2 previous similar messages [10862.276589] Lustre: server umount lustre-OST0000 complete [10866.146904] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10871.172686] Lustre: server umount lustre-MDT0000 complete [10874.153853] LustreError: 331566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629430 with bad export cookie 18295041006450448922 [10874.160723] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10874.161504] LustreError: 331566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10874.448254] Lustre: server umount lustre-MDT0001 complete [10883.591590] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [10885.645277] Key type lgssc unregistered [10885.904232] LNet: 332822:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10885.916564] LNetError: 332822:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10885.925772] LNet: Removed LNI 192.168.206.138@tcp [10886.535099] Key type .llcrypt unregistered [10886.537106] Key type ._llcrypt unregistered [10927.193821] Key type ._llcrypt registered [10927.196709] Key type .llcrypt registered [10927.299668] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [10928.144118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10928.178969] alg: No test for adler32 (adler32-zlib) [10929.110779] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [10929.304300] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [10930.961167] Key type lgssc registered [10931.881942] Lustre: Echo OBD driver; http://www.lustre.org/ [10939.559847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10939.581402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10940.942687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10944.164137] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10950.118552] LustreError: 335105:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10950.132031] LustreError: 335105:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10950.660130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10950.964925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10954.109435] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10955.834480] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10960.408491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10960.778265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10965.768071] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10965.926266] LustreError: 336595:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10965.945965] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10971.112351] LustreError: 336596:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10973.098061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10973.386751] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10975.500895] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10980.229558] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [10988.698739] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10996.504835] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11007.457792] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11007.470090] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11007.489162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11011.561878] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11011.567378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11011.590015] Lustre: Skipped 1 previous similar message [11011.606169] Lustre: Skipped 3 previous similar messages [11013.508136] Lustre: server umount lustre-MDT0000 complete [11016.676599] LustreError: 335850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11016.696313] LustreError: 335850:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11017.398314] LustreError: 335086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629573 with bad export cookie 10187118816854742826 [11017.409476] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11017.418245] LustreError: 335086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11017.781594] Lustre: server umount lustre-MDT0001 complete [11031.118380] Lustre: server umount lustre-OST0000 complete [11044.885938] Lustre: server umount lustre-OST0001 complete [11059.636665] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [11068.083962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11068.371896] LustreError: 340372:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.483533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11072.848182] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11078.632924] LustreError: 340373:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11078.647271] LustreError: 340373:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11080.718776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11081.180515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11085.879981] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11089.163663] Lustre: 341513:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11095.997723] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11102.181347] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11106.611757] LustreError: 341865:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11110.456815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11110.756910] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11110.767258] Lustre: Skipped 1 previous similar message [11113.847243] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11113.853139] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11116.776448] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11124.654684] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11128.251846] Lustre: 343383:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11134.442575] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11134.461550] Lustre: Skipped 2 previous similar messages [11134.468063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11134.477104] Lustre: Skipped 3 previous similar messages [11139.142862] Lustre: server umount lustre-MDT0000 complete [11139.557053] LustreError: 340367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11139.596537] LustreError: 340367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11143.363324] LustreError: 340353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629699 with bad export cookie 10187118816854744527 [11143.392708] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11143.763189] Lustre: server umount lustre-MDT0001 complete [11158.447148] Lustre: server umount lustre-OST0000 complete [11160.543827] Lustre: 334367:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788629700/real 1788629700] req@ffff8a0c01ec1f80 x1875513950972800/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788629716 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11160.586725] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11160.611631] Lustre: Skipped 2 previous similar messages [11162.216056] Lustre: server umount lustre-OST0001 complete [11165.400576] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11166.758167] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11168.158719] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:35:22 (1788629722) [11188.751598] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [11196.870933] loop0: detected capacity change from 0 to 1099511627776 [11197.401282] SGI XFS with ACLs, security attributes, quota, no debug enabled [11197.407951] *** VALIDATE xfs *** [11197.419907] XFS (loop0): Mounting V5 Filesystem [11197.450689] XFS (loop0): Ending clean mount [11198.621178] loop1: detected capacity change from 0 to 18691697672192 [11291.117907] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [11300.467228] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [11334.716433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11334.941101] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11334.965955] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11335.050418] Lustre: lustre-MDT0000: new disk, initializing [11335.112312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11335.125781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11339.142993] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11348.303821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11348.430634] Lustre: 348516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11348.503884] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11348.508030] Lustre: Skipped 1 previous similar message [11348.577270] Lustre: lustre-MDT0001: new disk, initializing [11348.673376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11348.683790] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11352.432566] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11360.785062] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11365.812811] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11371.427478] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11371.691961] Lustre: lustre-OST0000: new disk, initializing [11371.698593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11371.704795] Lustre: 350484:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11373.710916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11373.725968] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11373.840378] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11377.591609] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11386.968431] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11391.782257] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11392.038697] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11396.350486] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11396.576294] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11399.656808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11399.657699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11399.663964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11399.667490] Lustre: Skipped 1 previous similar message [11399.690964] LustreError: Skipped 1 previous similar message [11404.451665] Lustre: server umount lustre-OST0000 complete [11409.889224] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11409.892610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11409.909480] Lustre: Skipped 1 previous similar message [11409.921977] Lustre: Skipped 2 previous similar messages [11419.113152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11419.120475] Lustre: Skipped 3 previous similar messages [11422.175222] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11424.227370] LustreError: 350492:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11424.240296] LustreError: 350492:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11424.317730] Lustre: server umount lustre-MDT0000 complete [11427.685953] LustreError: 348509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788629983 with bad export cookie 10187118816854746081 [11427.690408] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11427.697240] LustreError: 348509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [11428.013267] Lustre: server umount lustre-MDT0001 complete [11434.898612] XFS (loop0): Unmounting Filesystem [11438.168214] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:39:52 (1788629992) [11443.747599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11444.240676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11444.249990] Lustre: Skipped 2 previous similar messages [11448.005319] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11455.539082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11459.818614] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11466.444872] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11471.525961] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11478.032262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11484.140347] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11490.822591] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11500.004059] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11500.008189] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11500.021537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11500.036692] Lustre: Skipped 1 previous similar message [11504.580512] Lustre: server umount lustre-OST0000 complete [11514.954474] Lustre: server umount lustre-MDT0000 complete [11518.447801] LustreError: 353022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788630074 with bad export cookie 10187118816854746963 [11518.454588] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11518.460293] LustreError: 353022:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11518.743582] Lustre: server umount lustre-MDT0001 complete [11528.721652] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [11531.254066] Key type lgssc unregistered [11531.500664] LNet: 356668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11531.508779] LNetError: 356668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11531.528625] LNet: Removed LNI 192.168.206.138@tcp [11532.387156] Key type .llcrypt unregistered [11532.392966] Key type ._llcrypt unregistered [11557.200549] Key type ._llcrypt registered [11557.202436] Key type .llcrypt registered [11557.304242] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [11558.064588] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11558.128446] alg: No test for adler32 (adler32-zlib) [11559.128329] Lustre: Lustre: Build Version: 2.17.58_3_g10061d6 [11559.303358] LNet: Added LNI 192.168.206.138@tcp [8/256/0/180] [11560.943228] Key type lgssc registered [11561.932211] Lustre: Echo OBD driver; http://www.lustre.org/ [11568.360451] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11568.374285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11569.659655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11572.887605] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11579.627462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11579.871179] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11583.329217] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11589.435429] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11594.687835] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11600.953234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11601.321384] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11608.164881] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11615.209034] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11621.858983] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11621.864721] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11621.888609] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11625.963160] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11625.963893] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11625.988898] Lustre: Skipped 2 previous similar messages [11627.203102] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.38@tcp (stopping) [11627.922243] Lustre: server umount lustre-MDT0001 complete [11631.071978] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11631.097350] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11632.307969] LustreError: 358225:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11636.199032] LustreError: 358230:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11636.224165] LustreError: 358230:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11637.731409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11637.846140] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11637.850634] Lustre: Found index 1 for lustre-MDT0001, updating log [11637.887587] Lustre: 358220:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11638.201051] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11642.721677] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11643.381043] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11643.897589] LustreError: 361748:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11643.912874] LustreError: 361748:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11643.925944] LustreError: 358222:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11643.991157] LustreError: 361753:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.138@tcp: cfg command failed: rc = -17 [11643.995459] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11643.995459] [11652.825971] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.38@tcp (stopping) [11653.603119] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11653.607277] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11656.641591] Lustre: server umount lustre-MDT0001 complete [11657.912762] LustreError: 359477:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11657.936347] LustreError: 359477:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11663.030398] LustreError: 358226:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11663.045739] LustreError: 358226:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11665.791676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11665.866785] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11665.870338] Lustre: Found index 1 for lustre-MDT0001, updating log [11665.900739] Lustre: 358219:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11666.267455] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11670.370784] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11671.534237] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11673.600437] LustreError: 362905:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11673.609821] LustreError: 362905:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11673.618284] LustreError: 358222:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11673.622825] LustreError: 358222:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11673.660328] LustreError: 362908:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11673.665567] LustreError: 362908:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11673.670385] LustreError: 362908:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.138@tcp: cfg command failed: rc = -17 [11673.677868] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.138@tcp [11673.677868] [11978.721352] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11978.731525] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11978.739117] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11978.744195] Lustre: Skipped 2 previous similar messages [11978.755019] Lustre: Skipped 3 previous similar messages [11982.520120] Lustre: server umount lustre-MDT0001 complete [11983.850213] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11983.871817] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11993.004365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11993.061718] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11993.064445] Lustre: Found index 1 for lustre-MDT0001, updating log [11993.086573] Lustre: 358220:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11993.379825] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11996.877265] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [11998.706985] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11999.732589] LustreError: 372078:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11999.738187] LustreError: 372078:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11999.745953] LustreError: 358222:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11999.751224] LustreError: 358222:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11999.804901] LustreError: 372081:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11999.814542] LustreError: 372081:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.138@tcp: cfg command failed: rc = -114 [11999.820192] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11999.820192] [12321.249651] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12321.249774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12321.251677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12321.251683] Lustre: Skipped 1 previous similar message [12321.280975] Lustre: Skipped 1 previous similar message [12327.501891] Lustre: server umount lustre-MDT0000 complete [12331.502229] LustreError: 358230:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12331.516228] LustreError: 358230:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12331.574842] LustreError: 358963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788630887 with bad export cookie 16706459107453448387 [12331.583482] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12331.594031] LustreError: 358963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12336.609219] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12336.624393] Lustre: Skipped 1 previous similar message [12338.012511] Lustre: server umount lustre-MDT0001 complete [12342.170476] Lustre: server umount lustre-OST0000 complete [12352.300532] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [12363.033249] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [12403.930752] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 13:55:57 (1788630957) [12410.755682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12411.022408] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12411.064693] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12411.157154] Lustre: lustre-MDT0000: new disk, initializing [12411.279878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12411.293839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12415.849525] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [12426.906883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12427.049845] Lustre: 383358:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12427.100317] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12427.104119] Lustre: Skipped 1 previous similar message [12427.234690] Lustre: lustre-MDT0001: new disk, initializing [12427.358581] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12427.414704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12427.430258] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12431.801412] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [12440.539438] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12445.848607] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12454.202362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12454.410238] Lustre: lustre-OST0000: new disk, initializing [12454.412426] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12454.425361] Lustre: 385327:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12454.529267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12460.653278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12460.665935] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12460.811332] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12461.480648] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_default_debug -1 all [12472.583085] Lustre: DEBUG MARKER: oleg638-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12481.248187] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12481.257687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12481.269701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12481.275841] Lustre: Skipped 6 previous similar messages [12487.605358] Lustre: server umount lustre-MDT0000 complete [12491.749111] LustreError: 384523:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12491.776475] LustreError: 384523:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12492.100308] LustreError: 383353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788631048 with bad export cookie 16706459107453450340 [12492.118054] LustreError: MGC192.168.206.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12492.122307] LustreError: 383353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12492.134678] LustreError: 384167:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12492.712061] Lustre: server umount lustre-MDT0001 complete [12508.163860] Lustre: server umount lustre-OST0000 complete [12541.101925] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing set_hostid [12553.408678] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing load_modules_local [12584.922434] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12285 sec ========= 13:58:58 (1788631138) [12586.999714] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:59:00 (1788631140) === [12591.032193] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:59:04 (1788631144) === [12635.205822] Lustre: DEBUG MARKER: oleg638-server.virtnet: executing unload_modules_local [12638.398537] Key type lgssc unregistered [12638.729457] LNet: 391698:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12638.735638] LNetError: 391698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12638.769108] LNet: Removed LNI 192.168.206.138@tcp [12639.957823] Key type .llcrypt unregistered [12639.960030] Key type ._llcrypt unregistered