[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 560832843 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002319] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004010] kvm-guest: setup PV IPIs [ 0.007631] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.008018] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.009010] pid_max: default: 32768 minimum: 301 [ 0.010116] LSM: Security Framework initializing [ 0.011037] Yama: becoming mindful. [ 0.012028] SELinux: Initializing. [ 0.013053] *** VALIDATE selinux *** [ 0.020540] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026657] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027131] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029083] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031083] *** VALIDATE tmpfs *** [ 0.032397] *** VALIDATE proc *** [ 0.033189] *** VALIDATE cgroup *** [ 0.034005] *** VALIDATE cgroup2 *** [ 0.035231] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036127] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038023] Spectre V2 : User space: Vulnerable [ 0.039006] Speculative Store Bypass: Vulnerable [ 0.042445] debug: unmapping init [mem 0xffffffffac059000-0xffffffffac060fff] [ 0.044137] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045594] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046020] ... version: 2 [ 0.047008] ... bit width: 48 [ 0.048009] ... generic registers: 4 [ 0.049007] ... value mask: 0000ffffffffffff [ 0.050014] ... max period: 00007fffffffffff [ 0.051009] ... fixed-purpose events: 3 [ 0.052008] ... event mask: 000000070000000f [ 0.053270] rcu: Hierarchical SRCU implementation. [ 0.055492] smp: Bringing up secondary CPUs ... [ 0.056560] x86: Booting SMP configuration: [ 0.057016] .... node #0, CPUs: #1 #2 #3 [ 0.060389] smp: Brought up 1 node, 4 CPUs [ 0.062011] smpboot: Max logical packages: 1 [ 0.063016] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.106313] node 0 deferred pages initialised in 41ms [ 0.110090] devtmpfs: initialized [ 0.112195] x86/mm: Memory block size: 128MB [ 0.114683] gcov: version magic: 0x41383552 [ 0.117418] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.121153] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.123300] pinctrl core: initialized pinctrl subsystem [ 0.124000] [ 0.124000] ************************************************************* [ 0.124007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.125000] ** ** [ 0.128009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.129008] ** ** [ 0.131008] ** This means that this kernel is built to expose internal ** [ 0.133010] ** IOMMU data structures, which may compromise security on ** [ 0.135009] ** your system. ** [ 0.137008] ** ** [ 0.139009] ** If you see this message and you are not debugging the ** [ 0.141008] ** kernel, report this immediately to your vendor! ** [ 0.143008] ** ** [ 0.145009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.147007] ************************************************************* [ 0.149686] NET: Registered protocol family 16 [ 0.150428] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.153051] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.155050] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.158122] cpuidle: using governor menu [ 0.159689] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.162587] PCI: Using configuration type 1 for base access [ 0.164176] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.174166] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.175013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.177141] cryptd: max_cpu_qlen set to 1000 [ 0.178020] ACPI: Added _OSI(Module Device) [ 0.179014] ACPI: Added _OSI(Processor Device) [ 0.180033] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.182009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.186775] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.193421] ACPI: Interpreter enabled [ 0.194047] ACPI: PM: (supports S0 S3 S4 S5) [ 0.195012] ACPI: Using IOAPIC for interrupt routing [ 0.196081] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.197355] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.206209] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.208030] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.210014] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.213070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.219257] acpiphp: Slot [2] registered [ 0.221124] acpiphp: Slot [5] registered [ 0.222095] acpiphp: Slot [6] registered [ 0.223106] acpiphp: Slot [7] registered [ 0.225108] acpiphp: Slot [8] registered [ 0.226098] acpiphp: Slot [9] registered [ 0.227174] acpiphp: Slot [10] registered [ 0.228104] acpiphp: Slot [3] registered [ 0.230055] acpiphp: Slot [4] registered [ 0.231072] acpiphp: Slot [11] registered [ 0.232067] acpiphp: Slot [12] registered [ 0.234064] acpiphp: Slot [13] registered [ 0.235099] acpiphp: Slot [14] registered [ 0.236070] acpiphp: Slot [15] registered [ 0.237066] acpiphp: Slot [16] registered [ 0.238060] acpiphp: Slot [17] registered [ 0.239064] acpiphp: Slot [18] registered [ 0.241064] acpiphp: Slot [19] registered [ 0.242090] acpiphp: Slot [20] registered [ 0.243065] acpiphp: Slot [21] registered [ 0.244069] acpiphp: Slot [22] registered [ 0.245189] acpiphp: Slot [23] registered [ 0.247068] acpiphp: Slot [24] registered [ 0.248125] acpiphp: Slot [25] registered [ 0.249062] acpiphp: Slot [26] registered [ 0.250079] acpiphp: Slot [27] registered [ 0.251124] acpiphp: Slot [28] registered [ 0.252074] acpiphp: Slot [29] registered [ 0.254090] acpiphp: Slot [30] registered [ 0.255081] acpiphp: Slot [31] registered [ 0.256190] PCI host bridge to bus 0000:00 [ 0.258014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.260014] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.261013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.264016] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.266015] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.268017] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.269208] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.271915] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.275210] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.284730] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.290060] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.291015] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.293012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.295012] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.297640] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.299898] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.302035] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.305553] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.310015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.324014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.328014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.335011] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.356018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.394013] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.432013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.442817] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.469015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.494012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.522014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.531709] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.542016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.547015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.560018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.569858] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.574014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.579018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.591018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.597914] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.603015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.608015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.619019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.627826] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.654012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.690013] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.761015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.795271] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.796301] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.798281] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.800277] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.801172] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.807035] iommu: Default domain type: Passthrough [ 0.808443] SCSI subsystem initialized [ 0.810126] ACPI: bus type USB registered [ 0.811082] usbcore: registered new interface driver usbfs [ 0.812079] usbcore: registered new interface driver hub [ 0.814070] usbcore: registered new device driver usb [ 0.815162] pps_core: LinuxPPS API ver. 1 registered [ 0.816010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.818055] PTP clock support registered [ 0.820082] EDAC MC: Ver: 3.0.0 [ 0.822143] PCI: Using ACPI for IRQ routing [ 0.823836] NetLabel: Initializing [ 0.824010] NetLabel: domain hash size = 128 [ 0.825012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.827076] NetLabel: unlabeled traffic allowed by default [ 0.828155] vgaarb: loaded [ 0.829283] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.831013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.837000] clocksource: Switched to clocksource kvm-clock [ 0.944084] VFS: Disk quotas dquot_6.6.0 [ 0.945471] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.947280] *** VALIDATE ramfs *** [ 0.948216] *** VALIDATE hugetlbfs *** [ 0.949319] pnp: PnP ACPI init [ 0.951480] pnp: PnP ACPI: found 6 devices [ 0.966962] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.969454] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.970967] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.972477] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.974111] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.975765] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.977848] NET: Registered protocol family 2 [ 0.979918] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.983907] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.986631] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.990679] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.993375] TCP: Hash tables configured (established 65536 bind 65536) [ 0.995650] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.997910] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.999916] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.001972] NET: Registered protocol family 1 [ 1.007443] RPC: Registered named UNIX socket transport module. [ 1.009937] RPC: Registered udp transport module. [ 1.011888] RPC: Registered tcp transport module. [ 1.013844] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.016477] NET: Registered protocol family 44 [ 1.018281] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.020742] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.023160] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.025687] PCI: CLS 0 bytes, default 64 [ 1.028694] Unpacking initramfs... [ 2.398482] debug: unmapping init [mem 0xffff8f6ffcc54000-0xffff8f6ffffbffff] [ 2.402035] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.403824] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.406257] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.965727] Initialise system trusted keyrings [ 2.967790] Key type blacklist registered [ 2.970105] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.979643] zbud: loaded [ 2.982508] *** VALIDATE nfs *** [ 2.984235] *** VALIDATE nfs4 *** [ 2.986280] pstore: using deflate compression [ 2.991125] Platform Keyring initialized [ 3.096994] NET: Registered protocol family 38 [ 3.100645] Key type asymmetric registered [ 3.102021] Asymmetric key parser 'x509' registered [ 3.103504] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.106485] io scheduler mq-deadline registered [ 3.108199] io scheduler kyber registered [ 3.109461] io scheduler bfq registered [ 3.111360] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.113843] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.117247] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.119493] ACPI: Power Button [PWRF] [ 3.126706] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.133638] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.207799] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.233182] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.249655] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.275408] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.301749] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.306757] Non-volatile memory driver v1.3 [ 3.307778] Linux agpgart interface v0.103 [ 3.341805] virtio_blk virtio1: [vda] 145872 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.344491] vda: detected capacity change from 0 to 74686464 [ 3.367895] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.370464] vdb: detected capacity change from 0 to 1073741824 [ 3.392141] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.395441] vdc: detected capacity change from 0 to 2621440000 [ 3.411958] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.415368] vdd: detected capacity change from 0 to 2621440000 [ 3.430429] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.433391] vde: detected capacity change from 0 to 4294967296 [ 3.448960] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.453293] vdf: detected capacity change from 0 to 4294967296 [ 3.463853] libphy: Fixed MDIO Bus: probed [ 3.469943] usbcore: registered new interface driver usbserial_generic [ 3.472683] usbserial: USB Serial support registered for generic [ 3.475382] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.481581] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.483587] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.487017] mousedev: PS/2 mouse device common for all mice [ 3.490188] rtc_cmos 00:05: RTC can wake from S4 [ 3.495110] rtc_cmos 00:05: registered as rtc0 [ 3.497173] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.497264] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.499026] intel_pstate: CPU model not supported [ 3.504601] hid: raw HID events driver (C) Jiri Kosina [ 3.508108] usbcore: registered new interface driver usbhid [ 3.509521] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.513595] usbhid: USB HID core driver [ 3.513776] drop_monitor: Initializing network drop monitor service [ 3.513902] Initializing XFRM netlink socket [ 3.515357] NET: Registered protocol family 10 [ 3.526305] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.531812] Segment Routing with IPv6 [ 3.542154] NET: Registered protocol family 17 [ 3.543772] mpls_gso: MPLS GSO support [ 3.550472] RAS: Correctable Errors collector initialized. [ 3.552705] AVX version of gcm_enc/dec engaged. [ 3.554398] AES CTR mode by8 optimization enabled [ 3.648887] sched_clock: Marking stable (3648866620, 0)->(4466119495, -817252875) [ 3.654197] registered taskstats version 1 [ 3.656510] Loading compiled-in X.509 certificates [ 3.658737] zswap: loaded using pool lzo/zbud [ 3.687710] Key type big_key registered [ 3.699908] Key type encrypted registered [ 3.701773] ima: No TPM chip found, activating TPM-bypass! [ 3.703777] ima: Allocated hash algorithm: sha1 [ 3.705279] ima: No architecture policies found [ 3.706598] evm: Initialising EVM extended attributes: [ 3.708060] evm: security.selinux [ 3.708873] evm: security.ima [ 3.709769] evm: security.capability [ 3.710746] evm: HMAC attrs: 0x1 [ 3.712724] rtc_cmos 00:05: setting system clock to 2026-07-31 03:17:17 UTC (1785467837) [ 3.718251] debug: unmapping init [mem 0xffffffffad003000-0xffffffffad1fffff] [ 3.720862] debug: unmapping init [mem 0xffffffffabd82000-0xffffffffac058fff] [ 3.727243] Write protecting the kernel read-only data: 28672k [ 3.730741] debug: unmapping init [mem 0xffffffffaa403000-0xffffffffaa5fffff] [ 3.733045] debug: unmapping init [mem 0xffffffffaad14000-0xffffffffaadfffff] [ 3.763040] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.775281] systemd[1]: Detected virtualization kvm. [ 3.777879] systemd[1]: Detected architecture x86-64. [ 3.780540] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.806165] systemd[1]: No hostname configured. [ 3.807974] systemd[1]: Set hostname to . [ 3.810651] random: systemd: uninitialized urandom read (16 bytes read) [ 3.813308] systemd[1]: Initializing machine ID from random generator. [ 3.861781] random: ln: uninitialized urandom read (6 bytes read) [ 3.932837] random: systemd: uninitialized urandom read (16 bytes read) [ 3.936838] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.940342] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.944548] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service.[ 4.148452] random: fast init done [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.499777] device-mapper: uevent: version 1.0.3 [ 4.502436] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.181386] virtio_net virtio0 ens2: renamed from eth0 [ 5.204347] scsi host0: ata_piix [ 5.224918] scsi host1: ata_piix [ 5.227066] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.229224] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.823396] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.214746] random: crng init done [ 10.216347] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.498752] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.590039] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.920134] SELinux: Disabled at runtime. [ 11.991212] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.001969] systemd[1]: Detected virtualization kvm. [ 12.004043] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.634715] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.637441] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.644104] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.648308] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.653166] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.662681] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.668955] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. Mounting POSIX Message Queue File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ 12.822537] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.136085] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.409639] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.446684] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.588330] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.605399] EDAC sbridge: Ver: 1.1.2 [ 15.336461] Key type dns_resolver registered [ 15.643521] NFS: Registering the id_resolver key type [ 15.645264] Key type id_resolver registered [ 15.646510] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg405-server login: [ 40.688956] libcfs: loading out-of-tree module taints kernel. [ 40.711924] Key type ._llcrypt registered [ 40.713599] Key type .llcrypt registered [ 40.768445] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 53.258480] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 55.366228] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 55.377423] alg: No test for adler32 (adler32-zlib) [ 56.473044] hrtimer: interrupt took 4035685 ns [ 56.849967] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 57.688441] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 59.519420] Key type lgssc registered [ 61.389229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.800420] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 139.283788] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 157.638784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 157.698843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 159.329328] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 159.441808] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 159.860832] Lustre: lustre-MDT0000: new disk, initializing [ 160.082309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 160.126624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 167.380772] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 185.444790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 185.577087] Lustre: 6489:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 185.614452] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 185.621608] Lustre: Skipped 1 previous similar message [ 185.755769] Lustre: lustre-MDT0001: new disk, initializing [ 185.840075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 185.880601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 185.892583] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 190.999800] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.963989] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 212.089000] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 212.784830] Lustre: lustre-OST0000: new disk, initializing [ 212.803993] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 212.818199] Lustre: 8427:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 212.958851] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 220.279148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 220.306617] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 220.477905] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 225.743116] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.222781] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 245.361434] Lustre: lustre-OST0001: new disk, initializing [ 245.366727] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 245.371127] Lustre: 9500:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 245.439672] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 253.514208] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.569680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 255.588357] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 255.665064] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 267.855485] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 276.579049] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 286.280550] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing check_logdir /tmp/testlogs/ [ 293.941776] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing yml_node [ 300.241694] Lustre: DEBUG MARKER: Client: 2.17.56.1 [ 304.165866] Lustre: DEBUG MARKER: MDS: 2.17.56.1 [ 307.708307] Lustre: DEBUG MARKER: OSS: 2.17.56.1 [ 310.138589] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jul 30 23:22:20 EDT 2026 [ 335.790854] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 338.621908] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 359.903942] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 359.923977] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 359.965268] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 363.507201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 363.520385] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 363.550470] Lustre: Skipped 1 previous similar message [ 365.762373] Lustre: server umount lustre-MDT0000 complete [ 370.361425] LustreError: 6481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785468204 with bad export cookie 11334044084946704732 [ 370.369418] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 370.381680] LustreError: 6481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 373.733806] LustreError: 8601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 373.734398] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 373.742343] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 373.742445] Lustre: Skipped 3 previous similar messages [ 373.762062] LustreError: 8601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 373.811555] Lustre: Skipped 2 previous similar messages [ 375.776587] LustreError: 8601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 375.780541] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 375.816911] LustreError: 8601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 375.837976] Lustre: Skipped 2 previous similar messages [ 377.386222] Lustre: server umount lustre-MDT0001 complete [ 383.274696] Lustre: server umount lustre-OST0000 complete [ 388.340375] Lustre: server umount lustre-OST0001 complete [ 399.061789] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 412.458989] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 467.132378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 467.458466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 467.483413] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 467.566385] Lustre: lustre-MDT0000: new disk, initializing [ 467.620557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 467.639788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 474.324407] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 486.741455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 486.839737] Lustre: 15065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 486.891689] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 486.896404] Lustre: Skipped 1 previous similar message [ 486.955712] Lustre: lustre-MDT0001: new disk, initializing [ 487.036885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 487.072481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 487.090378] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 493.160561] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 507.286548] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 516.984569] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 526.704390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 527.386320] Lustre: lustre-OST0000: new disk, initializing [ 527.398893] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 527.422181] Lustre: 17035:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 527.665855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 528.605743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 528.626900] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 528.809835] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 537.420166] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 551.559742] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 558.677508] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 559.010886] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 565.377679] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 565.821777] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 569.834606] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 569.839605] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 569.872400] Lustre: Skipped 1 previous similar message [ 569.878045] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 569.911581] Lustre: Skipped 1 previous similar message [ 575.663848] Lustre: server umount lustre-OST0000 complete [ 584.160987] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 584.175752] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 584.209809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 584.214337] Lustre: Skipped 2 previous similar messages [ 588.540595] Lustre: server umount lustre-MDT0000 complete [ 590.310149] LustreError: 15942:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 590.349592] LustreError: 15942:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 594.011553] LustreError: 15057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785468427 with bad export cookie 11334044084946706265 [ 594.026570] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 594.027455] LustreError: 15057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 595.011440] Lustre: server umount lustre-MDT0001 complete [ 602.338938] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 23:27:12 (1785468432) [ 610.081028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 610.691443] LustreError: 19302:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 610.811522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 615.912198] LustreError: 19303:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 617.029506] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 626.154136] LustreError: 19303:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 626.179543] LustreError: 19303:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 629.024923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 636.378691] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 649.449300] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 657.034574] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 667.199945] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 676.235215] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 685.578521] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 690.309062] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 700.896785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 700.901985] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 700.918367] Lustre: Skipped 1 previous similar message [ 700.925932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 700.933665] Lustre: Skipped 2 previous similar messages [ 706.465847] Lustre: server umount lustre-MDT0000 complete [ 706.539116] LustreError: 20203:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 711.902441] LustreError: 19284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785468545 with bad export cookie 11334044084946707147 [ 711.921833] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 712.403751] Lustre: server umount lustre-MDT0001 complete [ 726.801522] Lustre: server umount lustre-OST0000 complete [ 755.163754] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 771.563331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 771.800965] LustreError: 23712:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 771.816585] LustreError: 23712:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 771.888468] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 771.898655] Lustre: Skipped 2 previous similar messages [ 778.430035] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 789.719689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 796.037844] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 799.969498] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 809.549639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 820.433819] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 833.129285] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 833.353948] Lustre: lustre-OST0001: new disk, initializing [ 833.357526] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 833.362484] Lustre: 26029:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 834.611454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 834.626449] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 834.827689] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 841.774454] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 854.526797] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 860.133763] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 881.125708] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 881.128758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 881.159596] Lustre: Skipped 3 previous similar messages [ 881.186426] Lustre: Skipped 6 previous similar messages [ 884.969982] Lustre: server umount lustre-MDT0000 complete [ 886.240863] LustreError: 24442:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 886.296776] LustreError: 24442:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 891.009930] LustreError: 23693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785468724 with bad export cookie 11334044084946707994 [ 891.029867] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 891.037622] LustreError: 23693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 892.412292] Lustre: server umount lustre-MDT0001 complete [ 898.566919] Lustre: server umount lustre-OST0000 complete [ 903.503550] Lustre: server umount lustre-OST0001 complete [ 922.260273] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 936.004905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 936.660051] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 936.666222] Lustre: Skipped 3 previous similar messages [ 942.586619] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 954.128012] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.977451] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 965.303716] Lustre: 30199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 974.233803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 983.210317] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 996.926922] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1006.987117] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1016.858834] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1021.630768] Lustre: 32081:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1031.648427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1031.657325] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1031.670987] Lustre: Skipped 4 previous similar messages [ 1031.679220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1038.075317] Lustre: server umount lustre-MDT0000 complete [ 1038.308318] LustreError: 31847:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1038.347222] LustreError: 31847:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [ 1044.769793] LustreError: 29031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785468878 with bad export cookie 11334044084946709289 [ 1044.784855] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1044.785514] LustreError: 29031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1045.360219] Lustre: server umount lustre-MDT0001 complete [ 1059.789377] Lustre: server umount lustre-OST0000 complete [ 1074.916915] Lustre: server umount lustre-OST0001 complete [ 1086.292979] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:35:17 (1785468917) [ 1128.252203] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1142.730185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1148.137976] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1158.599960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1164.137686] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1167.709327] Lustre: 36057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1175.845852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1185.109053] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1197.416315] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1197.845867] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1197.854212] Lustre: Skipped 6 previous similar messages [ 1206.853225] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1216.907483] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1221.451534] Lustre: 37936:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1230.819513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1230.834956] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1230.876961] Lustre: Skipped 3 previous similar messages [ 1230.888813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1230.891444] Lustre: Skipped 4 previous similar messages [ 1236.113576] Lustre: server umount lustre-MDT0000 complete [ 1242.375513] LustreError: 37938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785469076 with bad export cookie 11334044084946710395 [ 1242.379615] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1242.387755] LustreError: 37938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1242.873314] Lustre: server umount lustre-MDT0001 complete [ 1259.423995] Lustre: 3620:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785469077/real 1785469077] req@ffff8f6f42e0d180 x1872198779673856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785469093 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1260.809969] Lustre: server umount lustre-OST0000 complete [ 1262.815086] Lustre: 3623:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785469080/real 1785469080] req@ffff8f6f44ca7b80 x1872198779674112/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785469096 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1265.635175] Lustre: 3620:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785469083/real 1785469083] req@ffff8f6f44ca7800 x1872198779674368/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785469099 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1266.221060] Lustre: server umount lustre-OST0001 complete [ 1278.255681] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:38:29 (1785469109) [ 1298.800714] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1314.132061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1315.205224] LustreError: 40081:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1315.266804] LustreError: 40081:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 1323.071943] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1334.852704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1342.520662] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1349.357563] Lustre: 41234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1361.594301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1372.803764] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1383.822661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1392.481763] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1401.850557] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1407.167296] Lustre: 43115:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1408.275530] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1408.285561] Lustre: Skipped 1 previous similar message [ 1412.066603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1412.072402] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1412.090458] Lustre: Skipped 5 previous similar messages [ 1418.114251] Lustre: server umount lustre-MDT0000 complete [ 1422.460657] LustreError: 40062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785469256 with bad export cookie 11334044084946711494 [ 1422.480634] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1422.489817] LustreError: 40062:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1429.296431] Lustre: server umount lustre-MDT0001 complete [ 1438.882521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1454.149649] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1458.679305] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1464.504529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1467.962844] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1467.977156] Lustre: Skipped 1 previous similar message [ 1471.225838] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1481.998571] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1484.221593] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1492.132333] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1494.998548] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1503.715616] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1503.729517] Lustre: Skipped 17 previous similar messages [ 1509.642128] Lustre: server umount lustre-MDT0000 complete [ 1514.923641] LustreError: 40063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785469348 with bad export cookie 11334044084946712600 [ 1514.945198] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1514.947115] LustreError: 40063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1515.862177] Lustre: server umount lustre-MDT0001 complete [ 1532.902603] Lustre: server umount lustre-OST0000 complete [ 1534.367128] Lustre: 3622:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785469352/real 1785469352] req@ffff8f70412a9f80 x1872198779773824/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785469368 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1537.812500] Lustre: server umount lustre-OST0001 complete [ 1553.159781] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:43:03 (1785469383) [ 1575.573701] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1591.568736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1599.129979] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1611.143034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1618.351194] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1621.666122] Lustre: 48883:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1630.431827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1640.088705] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1651.988343] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1660.582368] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1673.179322] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1681.044722] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1703.913577] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1703.925143] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1703.952881] Lustre: Skipped 11 previous similar messages [ 1707.101360] Lustre: server umount lustre-MDT0000 complete [ 1712.122943] LustreError: 47711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785469545 with bad export cookie 11334044084946713419 [ 1712.137418] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1712.140041] LustreError: 47711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1712.974378] Lustre: server umount lustre-MDT0001 complete [ 1728.652755] Lustre: server umount lustre-OST0000 complete [ 1729.509407] Lustre: 3620:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785469547/real 1785469547] req@ffff8f707df7ca80 x1872198779839616/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785469563 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1735.067479] Lustre: server umount lustre-OST0001 complete [ 1748.859649] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 23:46:19 (1785469579) [ 1752.574623] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1755.407685] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 23:46:25 (1785469585) [ 1770.054120] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 1781.191862] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1818.017123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1818.506636] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1818.513576] Lustre: Skipped 1 previous similar message [ 1818.559424] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1818.762924] Lustre: lustre-MDT0000: new disk, initializing [ 1818.899923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1818.910878] Lustre: Skipped 10 previous similar messages [ 1818.963794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1824.902520] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1838.602797] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1847.020700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1847.335744] Lustre: lustre-OST0000: new disk, initializing [ 1847.340325] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1847.345536] Lustre: Skipped 1 previous similar message [ 1847.355255] Lustre: 54996:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1848.537790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1848.558559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1848.590815] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1855.001513] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1867.373823] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1871.046101] Lustre: DEBUG MARKER: create test files [ 1910.240836] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1916.069149] Lustre: server umount lustre-OST0000 complete [ 1923.051978] Lustre: server umount lustre-MDT0000 complete [ 1939.679272] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 1944.095218] Key type lgssc unregistered [ 1944.485964] LNet: 56626:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1944.510233] LNetError: 56626:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1945.573872] LNet: Removed LNI 192.168.204.105@tcp [ 1946.642342] Key type .llcrypt unregistered [ 1946.644868] Key type ._llcrypt unregistered [ 1982.636862] Key type ._llcrypt registered [ 1982.639359] Key type .llcrypt registered [ 1982.811962] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1984.564051] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1984.704659] alg: No test for adler32 (adler32-zlib) [ 1986.105523] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 1986.536828] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 1988.255172] Key type lgssc registered [ 1990.425668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2002.032126] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2002.076583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2003.961137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2010.350115] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2022.356047] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2033.781069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2034.334847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2044.071751] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2048.786332] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2048.998055] Lustre: 59366:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 2049.010388] Lustre: 59366:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 2049.018964] Lustre: 59366:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2049.030497] Lustre: 59366:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2049.042400] Lustre: 59366:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2049.055175] Lustre: 59366:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2054.659968] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2061.628368] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2068.739383] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2084.836380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2084.852355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2084.880345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2086.704441] Lustre: server umount lustre-OST0000 complete [ 2093.189478] Lustre: server umount lustre-MDT0000 complete [ 2112.543147] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 2116.142860] Key type lgssc unregistered [ 2116.396102] LNet: 60993:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2116.419631] LNetError: 60993:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2116.446430] LNet: Removed LNI 192.168.204.105@tcp [ 2117.396447] Key type .llcrypt unregistered [ 2117.401970] Key type ._llcrypt unregistered [ 2168.759207] Key type ._llcrypt registered [ 2168.766164] Key type .llcrypt registered [ 2168.947111] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 2170.929718] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2170.976536] alg: No test for adler32 (adler32-zlib) [ 2172.291172] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 2172.751182] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 2174.503161] Key type lgssc registered [ 2176.330782] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2187.518168] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2187.529654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2189.182715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2194.154243] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2204.794460] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2216.199349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2217.322515] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2230.002553] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2233.684271] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2233.807266] Lustre: 63834:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2233.822627] Lustre: 63834:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2233.840696] Lustre: 63834:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2233.849623] Lustre: 63834:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2233.861728] Lustre: 63834:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2233.872097] Lustre: 63834:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2240.882265] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2248.537754] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2264.546512] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2264.558527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2264.581031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2268.235670] Lustre: server umount lustre-OST0000 complete [ 2274.401338] Lustre: server umount lustre-MDT0000 complete [ 2289.981965] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 2293.509660] Key type lgssc unregistered [ 2293.804253] LNet: 65408:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2293.811673] LNetError: 65408:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2293.825874] LNet: Removed LNI 192.168.204.105@tcp [ 2294.960968] Key type .llcrypt unregistered [ 2294.962932] Key type ._llcrypt unregistered [ 2317.435393] Key type ._llcrypt registered [ 2317.440802] Key type .llcrypt registered [ 2317.566604] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 2337.943187] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 2339.482754] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2339.516146] alg: No test for adler32 (adler32-zlib) [ 2340.723856] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 2341.032382] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 2342.759641] Key type lgssc registered [ 2343.980961] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2389.273332] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2389.305370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2390.788916] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2390.860635] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2390.981389] Lustre: lustre-MDT0000: new disk, initializing [ 2391.193192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2391.271551] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2398.552371] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2412.616731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2412.859110] Lustre: 68566:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2413.005598] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2413.012159] Lustre: Skipped 1 previous similar message [ 2413.245736] Lustre: lustre-MDT0001: new disk, initializing [ 2413.446218] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2413.523126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2413.542136] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2418.994992] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2430.799385] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2438.663372] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2448.535136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2448.808296] Lustre: lustre-OST0000: new disk, initializing [ 2448.811410] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2448.815698] Lustre: 70534:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2448.932369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2454.609320] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2454.620812] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2454.768354] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2457.482968] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2470.890874] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2478.539334] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2478.736456] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2489.068111] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2489.869057] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2495.969364] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2495.975488] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2495.985023] LustreError: Skipped 1 previous similar message [ 2495.987099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2496.058068] Lustre: Skipped 1 previous similar message [ 2499.355818] Lustre: server umount lustre-OST0000 complete [ 2505.700404] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2505.716508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2505.743994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2505.753685] Lustre: Skipped 1 previous similar message [ 2511.329145] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2511.333711] Lustre: Skipped 3 previous similar messages [ 2512.058569] Lustre: server umount lustre-MDT0000 complete [ 2517.415140] LustreError: 68557:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785470351 with bad export cookie 3113822787500022794 [ 2517.435675] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2518.224414] Lustre: server umount lustre-MDT0001 complete [ 2532.095324] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 23:59:22 (1785470362) [ 2548.899905] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 2607.757908] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 2623.273508] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 2676.933110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2677.494690] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2677.629150] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2677.786292] Lustre: lustre-MDT0000: new disk, initializing [ 2678.006469] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2678.064188] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2686.349791] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2702.554775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2702.882432] Lustre: 76625:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2703.007215] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2703.020532] Lustre: Skipped 1 previous similar message [ 2703.177407] Lustre: lustre-MDT0001: new disk, initializing [ 2703.559168] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2703.682467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2703.721167] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2710.331635] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2722.475876] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2729.940232] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2738.771217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2739.286803] Lustre: lustre-OST0000: new disk, initializing [ 2739.308544] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2739.321550] Lustre: 78594:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2739.535135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2748.578167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2748.612632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2748.799793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2750.337678] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2764.879993] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2772.415416] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2772.737589] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2779.568840] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2779.815332] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2784.746685] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2784.757567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2784.759101] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2784.763808] LustreError: Skipped 1 previous similar message [ 2784.807694] Lustre: Skipped 2 previous similar messages [ 2788.423277] Lustre: server umount lustre-OST0000 complete [ 2794.979040] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2794.986952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2795.004728] Lustre: Skipped 1 previous similar message [ 2795.024039] Lustre: Skipped 2 previous similar messages [ 2800.106167] LustreError: 76637:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2800.130067] LustreError: 76637:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2800.453817] Lustre: server umount lustre-MDT0000 complete [ 2805.217334] LustreError: 76637:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2805.231692] LustreError: 76637:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2805.428440] LustreError: 76616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785470639 with bad export cookie 3113822787500023676 [ 2805.429055] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2805.451186] LustreError: 76616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2806.127598] Lustre: server umount lustre-MDT0001 complete [ 2818.846359] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 00:04:09 (1785470649) [ 2827.838644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2828.690747] LustreError: 81011:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2828.787550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2833.893452] LustreError: 81012:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2834.785492] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2839.027530] LustreError: 81011:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2847.801285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2853.183182] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2863.650717] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2871.984229] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2880.998371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2881.543161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2881.549747] Lustre: Skipped 1 previous similar message [ 2890.476725] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2899.887259] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2905.304962] Lustre: 83690:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 8153a688-2593-4f40-8d4d-6afa3d2eb296 at adminstrative request [ 2910.185328] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2910.197932] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2919.399798] Lustre: MGS: Client 8153a688-2593-4f40-8d4d-6afa3d2eb296 (at 0@lo) reconnecting [ 2919.407871] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x2b368553d674a2ee to 0x2b368553d674a58e [ 2919.428178] Lustre: MGC192.168.204.105@tcp: Connection restored to 0@lo (at 0@lo) [ 2949.763379] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2950.058127] Lustre: lustre-OST0001: new disk, initializing [ 2950.069579] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2950.086296] Lustre: 84086:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2950.221589] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2958.085050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2958.103120] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2958.232269] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2960.256126] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 2972.923043] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2978.784972] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2978.790254] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2978.797279] LustreError: Skipped 1 previous similar message [ 2978.825142] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2978.834331] Lustre: Skipped 1 previous similar message [ 2982.700559] Lustre: server umount lustre-OST0001 complete [ 2983.918492] LustreError: 82841:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2983.942276] LustreError: 82841:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2989.031698] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2989.057710] Lustre: Skipped 2 previous similar messages [ 2999.276729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2999.282111] Lustre: Skipped 5 previous similar messages [ 3001.313810] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3001.738095] Lustre: server umount lustre-OST0000 complete [ 3006.947760] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3006.967606] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3012.850778] Lustre: server umount lustre-MDT0000 complete [ 3014.626014] LustreError: 83062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3014.650749] LustreError: 83062:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3018.451864] LustreError: 84085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785470852 with bad export cookie 3113822787500025230 [ 3018.456892] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3018.486159] LustreError: 84085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3019.224297] Lustre: server umount lustre-MDT0001 complete [ 3034.821200] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 00:07:44 (1785470864) [ 3038.708451] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3042.948192] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 00:07:52 (1785470872) [ 3045.934293] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3049.340551] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 00:07:59 (1785470879) [ 3052.503210] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3056.178812] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 00:08:05 (1785470885) [ 3059.338647] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3061.925885] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 00:08:12 (1785470892) [ 3065.025272] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3073.681899] loop: module loaded [ 3073.722811] loop0: detected capacity change from 0 to 4294967296 [ 3073.798920] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3075.596512] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3080.079756] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3085.571224] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3119.364401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3119.434561] LustreError: 88218:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3119.453485] LustreError: 88218:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3119.463834] LustreError: 88218:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3119.473473] LustreError: 88218:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3144.967098] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 3162.793620] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 3209.621463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3209.994373] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3210.031973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3210.175383] Lustre: lustre-MDT0000: new disk, initializing [ 3210.279108] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3210.329362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3215.298054] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3227.911889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3228.112318] Lustre: 91107:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3228.525100] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3233.762808] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3245.831135] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3254.154610] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3262.079418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3262.471433] Lustre: 93074:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3268.687406] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3268.883325] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3271.016961] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3282.318137] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3288.054795] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3288.279297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3294.179547] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3294.430397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3299.825418] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3299.828528] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3299.845032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3299.845042] Lustre: Skipped 3 previous similar messages [ 3299.849100] Lustre: Skipped 2 previous similar messages [ 3299.887138] LustreError: Skipped 1 previous similar message [ 3303.253249] Lustre: server umount lustre-OST0000 complete [ 3316.032975] Lustre: server umount lustre-MDT0000 complete [ 3319.278573] LustreError: 91114:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3319.310167] LustreError: 91114:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3321.705459] LustreError: 91099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785471155 with bad export cookie 3113822787500025846 [ 3321.712660] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3321.718985] LustreError: 91099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3322.347591] Lustre: server umount lustre-MDT0001 complete [ 3333.430880] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 00:12:44 (1785471164) [ 3343.789086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3349.947439] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3362.362620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3369.176967] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3380.042985] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3388.375798] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3397.904353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3406.339257] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3417.654645] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3425.159112] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3425.485920] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3432.078305] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3432.507641] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3441.161182] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3441.384458] Lustre: lustre-OST0001: new disk, initializing [ 3441.387416] Lustre: Skipped 2 previous similar messages [ 3441.391338] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3441.394527] Lustre: Skipped 3 previous similar messages [ 3441.400554] Lustre: 98862:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3448.400480] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3450.955294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3450.962615] Lustre: Skipped 2 previous similar messages [ 3450.969395] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3451.030464] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3459.826586] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3466.145414] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3466.396281] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3472.471954] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3472.856854] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3496.965869] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3535.957639] Lustre: Failing over lustre-MDT0000 [ 3536.884300] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3536.896882] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3536.909442] Lustre: Skipped 2 previous similar messages [ 3536.974984] LustreError: 96423:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3536.998356] LustreError: 96423:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3537.276283] Lustre: server umount lustre-MDT0000 complete [ 3554.784604] Lustre: 66787:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785471372/real 1785471372] req@00000000d6bd46df x1872201173488768/t0(0) o400->MGC192.168.204.105@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785471388 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3554.821732] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3566.105198] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3566.113125] LDISKFS-fs (dm-0): recovery complete [ 3566.121246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3579.365915] LustreError: 66786:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000057719427 x1872201173500800/t0(0) o250->MGC192.168.204.105@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3579.560546] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.5@tcp (not set up) [ 3579.575468] Lustre: Skipped 6 previous similar messages [ 3580.016211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3580.025990] Lustre: Skipped 6 previous similar messages [ 3580.125337] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3584.667360] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3584.676446] Lustre: 95488:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3585.020131] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3585.114695] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3585.464522] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3585.490573] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3585.513902] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3585.853639] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3585.884595] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3586.244890] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3586.251834] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3586.591412] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3586.610615] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 2 previous similar messages [ 3586.823912] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3586.944966] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3586.966847] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3587.306413] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3587.332877] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3588.759245] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3588.795399] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 3589.138713] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3589.167636] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3589.499381] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3589.510883] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3592.795577] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3592.833374] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 11 previous similar messages [ 3593.167960] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3593.192564] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 11 previous similar messages [ 3593.527710] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3593.548480] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 11 previous similar messages [ 3600.998561] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3601.021374] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 3601.378470] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3601.409743] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3601.741230] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3601.752511] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3617.159955] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3617.185516] Lustre: 101718:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 49 previous similar messages [ 3617.527081] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3617.543780] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3617.863168] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3617.878885] LustreError: 101718:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3644.111187] Lustre: 101718:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3644.130859] Lustre: 101718:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3644.142171] Lustre: 101718:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 9742f62b-e080-4154-9709-4635747129e1@192.168.204.5@tcp [ 3644.166120] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3644.176661] Lustre: 101718:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000503d43bb x1872201151226368/t0(8589934783) o101->9742f62b-e080-4154-9709-4635747129e1@192.168.204.5@tcp:678/0 lens 592/0 e 0 to 0 dl 1785471488 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3644.236649] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3644.264083] Lustre: lustre-MDT0000: Denying connection for new client 9742f62b-e080-4154-9709-4635747129e1 (at 192.168.204.5@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3644.273726] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3644.365250] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3644.369342] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3644.373060] Lustre: Skipped 2 previous similar messages [ 3644.443709] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 3644.444101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3668.963352] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3668.981964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3669.009140] Lustre: Skipped 3 previous similar messages [ 3682.275457] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3682.602865] Lustre: server umount lustre-OST0000 complete [ 3687.409820] LustreError: 102102:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3687.436847] LustreError: 102102:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 61 previous similar messages [ 3693.908502] Lustre: server umount lustre-OST0001 complete [ 3704.900826] Lustre: server umount lustre-MDT0000 complete [ 3710.945525] LustreError: 95473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785471544 with bad export cookie 3113822787500077058 [ 3710.949379] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3710.963253] LustreError: 95473:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3711.778049] Lustre: server umount lustre-MDT0001 complete [ 3725.299331] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 00:19:16 (1785471556) [ 3737.297502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3737.310820] Lustre: *** cfs_fail_loc=197, val=0*** [ 3753.055747] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3758.176193] LustreError: 103701:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.105@tcp: failed processing log, type 4: rc = -110 [ 3763.338860] Lustre: *** cfs_fail_loc=197, val=0*** [ 3763.340484] Lustre: Skipped 65 previous similar messages [ 3793.383534] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3804.256468] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3806.942983] Lustre: server umount lustre-OST0000 complete [ 3818.294760] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 00:20:49 (1785471649) [ 3843.660837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3851.763569] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 3865.074413] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3870.362198] LustreError: 105359:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3870.374977] LustreError: 105359:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3870.387307] LustreError: 105359:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 26, retries 0, failed: rc = -5 [ 3871.197537] Lustre: server umount lustre-MDT0000 complete [ 3917.504421] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 3935.456633] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 3994.615790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3995.067722] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3995.140871] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3995.543558] Lustre: lustre-MDT0000: new disk, initializing [ 3995.760601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4002.592853] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4016.423408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4016.559946] Lustre: 109564:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4016.947550] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4022.450966] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4039.173890] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4047.657452] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4055.919466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4056.157628] Lustre: 111537:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4063.919977] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4064.489798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4077.507446] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4083.649235] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4084.170282] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4090.165333] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4090.501544] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4095.467381] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4095.480740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4095.499887] LustreError: Skipped 4 previous similar messages [ 4095.501311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4095.501316] Lustre: Skipped 15 previous similar messages [ 4095.551052] Lustre: Skipped 6 previous similar messages [ 4099.945121] Lustre: server umount lustre-OST0000 complete [ 4111.113437] Lustre: server umount lustre-MDT0000 complete [ 4115.195383] LustreError: 109559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785471948 with bad export cookie 3113822787500087712 [ 4115.202039] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4115.208314] LustreError: 109559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4115.496324] Lustre: server umount lustre-MDT0001 complete [ 4128.421713] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:25:59 (1785471959) [ 4139.527450] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 4143.030553] Key type lgssc unregistered [ 4143.441758] LNet: 114025:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4143.449689] LNetError: 114025:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4143.488986] LNet: Removed LNI 192.168.204.105@tcp [ 4145.104539] Key type .llcrypt unregistered [ 4145.108362] Key type ._llcrypt unregistered [ 4169.101563] Key type ._llcrypt registered [ 4169.105097] Key type .llcrypt registered [ 4169.315579] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 4186.152168] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 4187.555701] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4187.614612] alg: No test for adler32 (adler32-zlib) [ 4188.765105] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 4189.191242] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 4190.887454] Key type lgssc registered [ 4191.920608] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4248.144361] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4248.185395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4249.568938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4249.595605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4249.702379] Lustre: lustre-MDT0000: new disk, initializing [ 4249.812566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4249.830427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4257.055047] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4273.370777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4273.887588] Lustre: lustre-OST0000: new disk, initializing [ 4273.902546] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4273.914502] Lustre: Skipped 1 previous similar message [ 4273.926156] Lustre: 118216:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4274.046571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4283.775943] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4284.013120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4284.040615] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4284.117770] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4298.060618] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4298.322220] Lustre: lustre-OST0001: new disk, initializing [ 4298.329810] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4298.341168] Lustre: 119081:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4298.454370] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4304.605946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4304.619499] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4304.681307] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4307.209359] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4320.228579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4320.225864] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4320.261540] Lustre: Skipped 1 previous similar message [ 4323.342456] Lustre: server umount lustre-MDT0000 complete [ 4332.436563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4338.726123] LustreError: 117400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785472172 with bad export cookie 12760612494584448724 [ 4338.731213] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4338.743319] LustreError: 117400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4339.140300] Lustre: server umount lustre-OST0000 complete [ 4344.331240] Lustre: server umount lustre-OST0001 complete [ 4355.922638] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 00:29:46 (1785472186) [ 4385.129175] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 00:30:16 (1785472216) [ 4400.528198] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 4435.440601] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:31:06 (1785472266) [ 4492.912942] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 4513.796639] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 4568.909546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4569.424273] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4569.501757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4569.721110] Lustre: lustre-MDT0000: new disk, initializing [ 4569.834752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4569.883526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4576.070529] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4590.366448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4590.581051] Lustre: 126879:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4590.685188] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4590.704531] Lustre: Skipped 1 previous similar message [ 4590.843993] Lustre: lustre-MDT0001: new disk, initializing [ 4590.985231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4591.044729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4591.067344] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4597.603852] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4617.396450] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4626.443840] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4637.465511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4637.963404] Lustre: lustre-OST0000: new disk, initializing [ 4637.971477] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4637.980592] Lustre: 128849:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4638.090680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4646.005933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4646.024985] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4646.187120] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4648.672914] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4661.848311] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4667.325969] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4667.525617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4674.445335] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4674.962768] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4682.212881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4682.214069] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4682.229251] LustreError: Skipped 1 previous similar message [ 4682.232391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4682.266932] Lustre: Skipped 1 previous similar message [ 4682.312926] Lustre: Skipped 2 previous similar messages [ 4684.452097] Lustre: server umount lustre-OST0000 complete [ 4692.449268] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4692.450251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4692.472293] Lustre: Skipped 1 previous similar message [ 4692.502204] Lustre: Skipped 1 previous similar message [ 4697.535620] Lustre: server umount lustre-MDT0000 complete [ 4702.700197] LustreError: 126891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4702.747061] LustreError: 126891:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4702.924795] LustreError: 126871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785472536 with bad export cookie 12760612494584449942 [ 4702.940380] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4702.961600] LustreError: 126871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4703.950176] Lustre: server umount lustre-MDT0001 complete [ 4716.728544] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:35:47 (1785472547) [ 4727.124234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4727.786285] LustreError: 131268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4727.989736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4733.410190] LustreError: 131269:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4734.383869] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4737.513033] LustreError: 131268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4742.625450] LustreError: 131269:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4745.185348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.779650] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4762.557876] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4772.008695] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4781.414113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4781.664049] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4781.673712] Lustre: Skipped 1 previous similar message [ 4789.936944] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4803.156186] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4809.748441] Lustre: *** cfs_fail_loc=159, val=0*** [ 4825.379593] Lustre: lustre-MDT0000: Client 2ee6c309-64ff-4d1d-b451-4dd82a0b20b2 (at 192.168.204.5@tcp) reconnecting [ 4828.029213] Lustre: *** cfs_fail_loc=159, val=0*** [ 4828.036059] Lustre: Skipped 5 previous similar messages [ 4843.232656] Lustre: lustre-MDT0000: Client 2ee6c309-64ff-4d1d-b451-4dd82a0b20b2 (at 192.168.204.5@tcp) reconnecting [ 4852.704273] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4852.713680] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4852.736863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4857.806747] Lustre: server umount lustre-OST0000 complete [ 4863.458427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4863.474071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4863.494279] Lustre: Skipped 1 previous similar message [ 4863.510079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4863.512341] Lustre: Skipped 2 previous similar messages [ 4869.156953] Lustre: server umount lustre-MDT0000 complete [ 4870.116154] LustreError: 131268:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4870.141752] LustreError: 131268:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4874.096242] LustreError: 133994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785472707 with bad export cookie 12760612494584450831 [ 4874.107647] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4874.117050] LustreError: 133994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4874.886677] Lustre: server umount lustre-MDT0001 complete [ 4890.529239] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 4893.885102] Key type lgssc unregistered [ 4894.300132] LNet: 135049:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4894.324955] LNetError: 135049:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4895.404221] LNet: Removed LNI 192.168.204.105@tcp [ 4896.824134] Key type .llcrypt unregistered [ 4896.825664] Key type ._llcrypt unregistered [ 4939.022727] Key type ._llcrypt registered [ 4939.027774] Key type .llcrypt registered [ 4939.165236] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 4941.563381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4941.587559] alg: No test for adler32 (adler32-zlib) [ 4942.946100] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 4943.220247] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 4944.864877] Key type lgssc registered [ 4945.766623] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4954.750253] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4954.775976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4956.128838] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4962.972260] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4965.346796] LustreError: 136467:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4965.359136] LustreError: 136467:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4970.465735] LustreError: 136468:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4975.596179] LustreError: 136467:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4976.580261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4977.489654] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4984.524173] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 4995.727478] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5003.717194] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5015.912521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5016.904572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5026.130121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5028.759984] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5039.430773] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5052.549137] Lustre: *** cfs_fail_loc=159, val=0*** [ 5069.022046] Lustre: lustre-MDT0000: Client 61fb0b9f-e4ab-4e54-9cd4-cb43fc5ed81c (at 192.168.204.5@tcp) reconnecting [ 5071.953594] Lustre: *** cfs_fail_loc=159, val=0*** [ 5087.519164] Lustre: lustre-MDT0001: Client 61fb0b9f-e4ab-4e54-9cd4-cb43fc5ed81c (at 192.168.204.5@tcp) reconnecting [ 5089.985572] Lustre: *** cfs_fail_loc=159, val=0*** [ 5089.997221] Lustre: Skipped 5 previous similar messages [ 5105.391257] Lustre: lustre-MDT0001: Client 61fb0b9f-e4ab-4e54-9cd4-cb43fc5ed81c (at 192.168.204.5@tcp) reconnecting [ 5115.511786] Lustre: 139506:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5119.181799] Lustre: *** cfs_fail_loc=159, val=0*** [ 5119.195562] Lustre: Skipped 3 previous similar messages [ 5135.583204] Lustre: lustre-MDT0000: Client 26b603d3-1a3b-492b-9abf-b4b50ee0d17e (at 192.168.204.5@tcp) reconnecting [ 5138.092578] Lustre: *** cfs_fail_loc=159, val=0*** [ 5138.094401] Lustre: Skipped 12 previous similar messages [ 5153.602030] Lustre: lustre-MDT0000: Client 26b603d3-1a3b-492b-9abf-b4b50ee0d17e (at 192.168.204.5@tcp) reconnecting [ 5155.645125] Lustre: 139800:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5177.850330] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5177.874729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5177.890614] Lustre: Skipped 1 previous similar message [ 5182.945434] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5182.948062] Lustre: Skipped 1 previous similar message [ 5183.755450] Lustre: server umount lustre-OST0000 complete [ 5192.173345] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5192.188371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5192.208159] Lustre: Skipped 1 previous similar message [ 5192.217396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5196.783665] Lustre: server umount lustre-MDT0000 complete [ 5198.319875] LustreError: 136464:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5198.345445] LustreError: 136464:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5201.400890] LustreError: 136448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785473035 with bad export cookie 17939560128525033407 [ 5201.404410] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5201.412867] LustreError: 136448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5202.001747] Lustre: server umount lustre-MDT0001 complete [ 5216.844694] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 5221.124611] Key type lgssc unregistered [ 5221.551598] LNet: 140881:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5221.560772] LNetError: 140881:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5221.599408] LNet: Removed LNI 192.168.204.105@tcp [ 5223.079468] Key type .llcrypt unregistered [ 5223.082449] Key type ._llcrypt unregistered [ 5258.900656] Key type ._llcrypt registered [ 5258.902109] Key type .llcrypt registered [ 5259.129992] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 5262.728517] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5262.787613] alg: No test for adler32 (adler32-zlib) [ 5264.585319] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 5265.540523] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 5267.639534] Key type lgssc registered [ 5269.632164] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5280.645874] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5280.699323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5282.101530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5287.919130] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5291.489088] LustreError: 142300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5291.519920] LustreError: 142300:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5296.611668] LustreError: 142301:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5299.138159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5299.471412] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5306.102360] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5316.457699] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5324.340103] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5332.639390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5333.185982] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5341.048538] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5344.504232] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5351.690568] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5353.560881] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5361.071194] Lustre: 144882:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5367.331773] Lustre: 144931:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5369.530876] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5370.716886] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5376.038659] Lustre: 145078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5377.117718] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5378.247299] Lustre: 145186:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5385.717710] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5385.740885] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5385.802676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5387.744780] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5387.744876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5387.759940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5391.366429] Lustre: server umount lustre-OST0000 complete [ 5402.088851] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5402.107143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5402.154608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5402.173032] Lustre: Skipped 1 previous similar message [ 5404.592583] Lustre: server umount lustre-MDT0000 complete [ 5408.224680] LustreError: 142297:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5408.248540] LustreError: 142297:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5409.907806] LustreError: 142282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785473243 with bad export cookie 8574623033884199309 [ 5409.923550] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5409.926200] LustreError: 142282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5410.432806] Lustre: server umount lustre-MDT0001 complete [ 5423.811522] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 5427.161951] Key type lgssc unregistered [ 5427.641228] LNet: 146242:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5427.656147] LNetError: 146242:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5428.727174] LNet: Removed LNI 192.168.204.105@tcp [ 5429.878575] Key type .llcrypt unregistered [ 5429.882662] Key type ._llcrypt unregistered [ 5461.908733] Key type ._llcrypt registered [ 5461.912336] Key type .llcrypt registered [ 5462.131067] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 5464.029059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5464.184140] alg: No test for adler32 (adler32-zlib) [ 5465.486806] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 5466.042041] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 5467.927745] Key type lgssc registered [ 5469.323635] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5480.161653] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5480.199444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5481.946633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5489.555551] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5492.195291] LustreError: 147660:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5492.227875] LustreError: 147660:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5497.313680] LustreError: 147659:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5501.192374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5502.003578] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5508.785321] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5519.444784] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5526.722699] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5536.540143] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5537.251942] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5547.513179] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5547.818171] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5552.639526] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5559.158970] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5565.379181] Lustre: *** cfs_fail_loc=159, val=0*** [ 5582.096351] Lustre: lustre-MDT0000: Client fe2bda7c-2e13-4e31-b35c-90b847da1e0c (at 192.168.204.5@tcp) reconnecting [ 5598.690212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5598.691160] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5598.700453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5598.717758] Lustre: Skipped 1 previous similar message [ 5601.069812] Lustre: server umount lustre-OST0000 complete [ 5608.934614] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5608.948167] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5608.955364] Lustre: Skipped 2 previous similar messages [ 5614.054325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5614.065322] Lustre: Skipped 1 previous similar message [ 5619.182577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5619.185136] Lustre: Skipped 1 previous similar message [ 5622.239350] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5622.557396] Lustre: server umount lustre-MDT0000 complete [ 5624.294775] LustreError: 147659:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5624.322930] LustreError: 147659:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5628.656065] LustreError: 147641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785473462 with bad export cookie 2171569855946878740 [ 5628.672451] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5628.675470] LustreError: 147641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5629.325811] Lustre: server umount lustre-MDT0001 complete [ 5649.108622] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 5653.689773] Key type lgssc unregistered [ 5654.126407] LNet: 151357:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5654.139497] LNetError: 151357:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5654.154813] LNet: Removed LNI 192.168.204.105@tcp [ 5655.144994] Key type .llcrypt unregistered [ 5655.148341] Key type ._llcrypt unregistered [ 5693.065747] Key type ._llcrypt registered [ 5693.075771] Key type .llcrypt registered [ 5693.294804] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 5695.617453] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5695.775881] alg: No test for adler32 (adler32-zlib) [ 5697.083585] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 5697.454072] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 5699.239898] Key type lgssc registered [ 5701.616223] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5711.270271] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5711.293339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5713.058705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5718.496187] LustreError: 152775:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5718.518181] LustreError: 152775:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5719.738261] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5722.594229] LustreError: 152776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5727.737846] LustreError: 152775:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5732.644510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5732.833957] LustreError: 152776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5733.192032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5739.495130] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5750.922804] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5760.456482] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5769.708562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5770.096508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5777.282166] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5778.691778] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 5782.403911] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5789.097751] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5811.859749] Lustre: lustre-MDT0001: evicting 0f99f5e1-b683-46df-a1e8-46c9bed3866f (at 192.168.204.5@tcp) by administrative request [ 5811.873262] Lustre: Skipped 1 previous similar message [ 5843.938724] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5843.940229] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5843.951714] LustreError: Skipped 1 previous similar message [ 5843.960269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5849.057817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5849.071808] Lustre: Skipped 1 previous similar message [ 5854.205273] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5854.214802] Lustre: Skipped 2 previous similar messages [ 5858.271376] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5858.633682] Lustre: server umount lustre-OST0000 complete [ 5864.418855] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5864.435958] Lustre: Skipped 1 previous similar message [ 5864.444138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5864.453149] Lustre: Skipped 1 previous similar message [ 5869.537123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5878.239142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5878.469830] Lustre: server umount lustre-MDT0000 complete [ 5879.787068] LustreError: 155691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5879.803201] LustreError: 155691:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5882.982322] LustreError: 152756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785473716 with bad export cookie 10804936296178273378 [ 5882.988691] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5882.996517] LustreError: 152756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5883.749231] Lustre: server umount lustre-MDT0001 complete [ 5898.999978] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 5903.228297] Key type lgssc unregistered [ 5903.674707] LNet: 156855:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5903.684638] LNetError: 156855:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5903.717368] LNet: Removed LNI 192.168.204.105@tcp [ 5905.032236] Key type .llcrypt unregistered [ 5905.036385] Key type ._llcrypt unregistered [ 6014.212154] Key type ._llcrypt registered [ 6014.223391] Key type .llcrypt registered [ 6014.399604] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 6016.743543] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6016.804568] alg: No test for adler32 (adler32-zlib) [ 6018.136758] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 6018.729139] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 6020.687160] Key type lgssc registered [ 6023.433189] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6035.577458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6035.595441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6037.279381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6044.092620] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6046.700223] LustreError: 159352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6046.737601] LustreError: 159352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6051.818945] LustreError: 159353:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6056.210165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6057.178535] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6065.573616] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6078.665224] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6086.611668] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6095.846876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6096.392763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6101.557628] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6105.525835] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6106.677442] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6115.897471] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6132.199412] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6132.202366] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6132.208171] LustreError: Skipped 1 previous similar message [ 6132.239134] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6132.242688] Lustre: Skipped 1 previous similar message [ 6132.261889] Lustre: Skipped 1 previous similar message [ 6136.189618] Lustre: server umount lustre-OST0000 complete [ 6147.563283] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6147.588044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6147.603275] Lustre: Skipped 1 previous similar message [ 6151.237872] Lustre: server umount lustre-MDT0000 complete [ 6152.672468] LustreError: 159347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6152.706118] LustreError: 159347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6156.745134] LustreError: 159334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785473990 with bad export cookie 15508378842816208704 [ 6156.758646] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6156.760608] LustreError: 159334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6157.742097] Lustre: server umount lustre-MDT0001 complete [ 6180.007220] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 6185.885301] Key type lgssc unregistered [ 6186.273680] LNet: 162943:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6186.292046] LNetError: 162943:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6186.319232] LNet: Removed LNI 192.168.204.105@tcp [ 6187.869198] Key type .llcrypt unregistered [ 6187.870739] Key type ._llcrypt unregistered [ 6223.384720] Key type ._llcrypt registered [ 6223.391270] Key type .llcrypt registered [ 6223.582440] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 6225.254294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6225.634063] alg: No test for adler32 (adler32-zlib) [ 6227.104493] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 6227.407536] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 6229.129307] Key type lgssc registered [ 6230.499476] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6252.982286] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:01:23 (1785474083) [ 6275.675986] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 6291.716347] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 6340.428993] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6340.475671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6341.919459] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6341.998612] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6342.088352] Lustre: lustre-MDT0000: new disk, initializing [ 6342.187076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6342.226207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6347.601560] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6361.036367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6361.256953] Lustre: 167247:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6361.359439] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6361.369868] Lustre: Skipped 1 previous similar message [ 6361.548422] Lustre: lustre-MDT0001: new disk, initializing [ 6361.722058] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6361.787248] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6361.802285] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6367.811513] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6381.028501] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6389.840206] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6400.722350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6401.223168] Lustre: lustre-OST0000: new disk, initializing [ 6401.229912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6401.240762] Lustre: 169216:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6401.354151] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6408.410501] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6408.779506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6408.791320] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6408.962189] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6420.183446] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6434.798366] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6434.805922] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6434.827627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6434.873241] Lustre: Skipped 1 previous similar message [ 6435.210619] Lustre: server umount lustre-OST0000 complete [ 6443.494978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6443.507232] LustreError: Skipped 1 previous similar message [ 6443.519330] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6443.550189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6443.552673] Lustre: Skipped 1 previous similar message [ 6445.024610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6445.025741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6445.052321] Lustre: Skipped 1 previous similar message [ 6446.883121] Lustre: server umount lustre-MDT0000 complete [ 6451.325467] LustreError: 167239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785474285 with bad export cookie 5810974793666158785 [ 6451.334431] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6451.338251] LustreError: 167239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6451.940051] Lustre: server umount lustre-MDT0001 complete [ 6467.803737] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 6471.941811] Key type lgssc unregistered [ 6472.366631] LNet: 171120:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6472.376555] LNetError: 171120:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6472.405595] LNet: Removed LNI 192.168.204.105@tcp [ 6473.544641] Key type .llcrypt unregistered [ 6473.550607] Key type ._llcrypt unregistered [ 6504.765188] Key type ._llcrypt registered [ 6504.767177] Key type .llcrypt registered [ 6504.902736] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 6506.107476] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6506.153074] alg: No test for adler32 (adler32-zlib) [ 6507.581295] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 6507.926216] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 6509.703179] Key type lgssc registered [ 6510.585605] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6518.687906] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6518.710688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6520.125188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6525.121579] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6529.521422] LustreError: 172540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6529.545440] LustreError: 172540:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6534.641233] LustreError: 172539:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6537.501239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6538.008197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6544.240832] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6554.707978] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6564.163321] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6575.194135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6576.009875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6585.595126] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 6589.373187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6594.863495] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6602.724165] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6602.771768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6602.798743] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6602.811032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.834331] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6604.017539] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6604.037250] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6604.064582] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6604.076315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.096782] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6605.430458] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6605.454031] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6605.466880] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6605.472338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.484645] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6606.696856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.939484] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6607.947194] Lustre: Skipped 1 previous similar message [ 6607.976115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6607.989964] Lustre: Skipped 1 previous similar message [ 6608.003908] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6608.014357] Lustre: Skipped 1 previous similar message [ 6608.025199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.045479] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6608.057050] Lustre: Skipped 1 previous similar message [ 6609.529983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.465764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.599857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.834211] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6612.847105] Lustre: Skipped 3 previous similar messages [ 6612.871826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6612.896895] Lustre: Skipped 3 previous similar messages [ 6612.910967] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6612.913675] Lustre: Skipped 3 previous similar messages [ 6612.924647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.955259] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6612.966871] Lustre: Skipped 3 previous similar messages [ 6614.235126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.495261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.675699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.922279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.121911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.224212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.312138] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6621.319740] Lustre: Skipped 6 previous similar messages [ 6621.349288] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6621.368834] Lustre: Skipped 6 previous similar messages [ 6621.374083] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6621.376746] Lustre: Skipped 6 previous similar messages [ 6621.383904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.399414] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6621.413200] Lustre: Skipped 6 previous similar messages [ 6622.656453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.294499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.640613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.216759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.440795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.830268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.431420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.581287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.787246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.863864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.051141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.363935] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6637.371396] Lustre: Skipped 11 previous similar messages [ 6637.395616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6637.403579] Lustre: Skipped 11 previous similar messages [ 6637.406881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6637.410936] Lustre: Skipped 11 previous similar messages [ 6637.417410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.426027] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6637.438507] Lustre: Skipped 11 previous similar messages [ 6638.209295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.072816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.901405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.035469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.136597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.099825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.302549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.319414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.272393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.157580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.209124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.317191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.492507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.552203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.649719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.718207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.785707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.984732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.284892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.198816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.136383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.266042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.482521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.860509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.023533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.554773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.346382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.861082] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6669.883040] Lustre: Skipped 27 previous similar messages [ 6669.928355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6669.951626] Lustre: Skipped 27 previous similar messages [ 6669.953551] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6669.956016] Lustre: Skipped 27 previous similar messages [ 6669.979305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.011487] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6670.026569] Lustre: Skipped 27 previous similar messages [ 6671.455892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.899151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.257355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.866647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.515618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.014503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.996060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.538283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.852126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.482321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.801396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.233026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.550104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.987060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.729412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.378187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.983345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.021737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.139976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.053429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.162741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.091680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.032148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.323437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.482131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.598982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.737440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.009872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.474568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.844321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.046216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.128564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.316750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.808660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.135272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.485499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.369283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.791590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.228869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.936956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.430709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.996596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.721340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.997502] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6735.010021] Lustre: Skipped 43 previous similar messages [ 6735.044707] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6735.088369] Lustre: Skipped 43 previous similar messages [ 6735.116914] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6735.137497] Lustre: Skipped 43 previous similar messages [ 6735.146365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.173491] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6735.192992] Lustre: Skipped 43 previous similar messages [ 6737.715709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.989847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.242305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.438474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.457949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.979140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.717402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.696976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.791309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.381466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.797040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.351678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.744491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.070824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.192474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.118456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.128076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.033152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.152636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.331228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.534887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.744454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.871723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.088366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.057258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.069545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.031144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.175858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.204230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.311649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.422795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.574868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.450871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.257498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.298132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.408924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.727747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.735409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.864246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.042459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.084559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.617301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.064044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.279452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.264157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.410406] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6793.425080] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6793.428315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.386800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.955422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.340477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.244170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.290675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.423209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.773345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.034114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.239197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.592533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.594926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.888932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.422442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.371964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.689907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.065558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.334501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.636476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.104248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.564915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.767268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.119947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.282606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.416806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.532307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.672166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.727385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.575557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.498727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.543384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.771348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.997148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.016335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.188086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.080208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.459233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.301906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.526295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.584810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.040315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.495965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.858168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.135531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.623039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.895125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.861918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.580325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.310724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.293625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.826477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.495808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.456079] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6863.476220] Lustre: Skipped 97 previous similar messages [ 6863.536899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6863.571417] Lustre: Skipped 97 previous similar messages [ 6863.589521] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6863.606561] Lustre: Skipped 97 previous similar messages [ 6863.627766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.701897] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6863.737750] Lustre: Skipped 97 previous similar messages [ 6865.612180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.178719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.606627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.615726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.899672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.354583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.578453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.672949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.185587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.491748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.600177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.239465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.488798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.613159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.850202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.844368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.882587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.279472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.616557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.001330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.013440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.862498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.502164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.340906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.887422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.099634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.871409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.983890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.531155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.843657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.212996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.067184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.357254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.668329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.019922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.156564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6916.556820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.880522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.966532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.228472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.485804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.979131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.805538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.408574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.947143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.449495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.758921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6931.900693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.149380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.411796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.572924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.073726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6938.468302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.011653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6941.594161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6943.175865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.344885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6945.693291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6947.353955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.491971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.036557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.186357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.779776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6953.741549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.532576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.719150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6956.740588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.665524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.616571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6959.982832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6961.156183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6962.844890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.269431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.679610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6967.115938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.619414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6970.218717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6971.720340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.731157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6973.668290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.665619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.823123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.075911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.264019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.244678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.341542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6981.373508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.614183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.866397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6984.987716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.316725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.380134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.571734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6989.824392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.888080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.029650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.181691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6994.405526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.425355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.695232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.924897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6999.177631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.460649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7001.499393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.516325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.800951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7004.909868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.963416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7007.040914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.307591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7009.358926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.419994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.504116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.773612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.071043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.168753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.397468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7017.785840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.018946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7020.216684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.238945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.313035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7023.592164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7024.705714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.931348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7026.976634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7028.607193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.851408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7031.168141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.541482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.037861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7035.703272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.065057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7038.433342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.560772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7041.005332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7042.576632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7043.854165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7045.452746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.610107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7047.885162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7048.843338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7050.362796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7051.435443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7052.383441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7053.468094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7054.721928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7055.912964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7057.015697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7058.019846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7059.374093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.464215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7061.776748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7062.967612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.139424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7065.398771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7066.733927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7067.938576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7069.303183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7070.903698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.324091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7073.977641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.474633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7076.847884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.240144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.427703] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7079.442921] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7079.448384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.965794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.105189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.406569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7084.889615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7086.196203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.580456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7089.123641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.696029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7092.246291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.392265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7094.702087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.225273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.766883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.121501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7100.466942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.676427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7103.125071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7104.457736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7105.645196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.724315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.072196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.461712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7110.593340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7111.851913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.902889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.073622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.131291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7116.192670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7117.258374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.146437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7119.171513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.078545] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7120.082102] Lustre: Skipped 197 previous similar messages [ 7120.091561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7120.097587] Lustre: Skipped 197 previous similar messages [ 7120.100994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7120.111808] Lustre: Skipped 197 previous similar messages [ 7120.114723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.122718] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7120.126451] Lustre: Skipped 197 previous similar messages [ 7120.977612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7122.495434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.786493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.957373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7126.170207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.234995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.227323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.263257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.383773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.481434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7132.737368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.017492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.986108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7136.023575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.034049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7138.105250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.153277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.293693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7141.359652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.550136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7143.522374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7145.043307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7146.497516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.647239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7148.508661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.866177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.953083] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7151.983410] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7152.010097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.701884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.392358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.064275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7159.035107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7160.834058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.581794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.420151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7165.975617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7168.028806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7169.591780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.241055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7173.042694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.698433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.159895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.370302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7178.339772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7179.782405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.223595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.685756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7183.488249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.376273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.301187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7186.583826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7187.625263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7188.955284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.071555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.142931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7192.136962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.233501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7194.251887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.306660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7196.255354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7197.410291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.497449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7199.601120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.761883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.010754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.250281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7204.293465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.524831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.915388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7208.161664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7209.434344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7210.665627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.922220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.166547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7214.425311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.758721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7217.249034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7218.665353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7219.942520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7221.206314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7222.427216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.772163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.066402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7226.311660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.617642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.946898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7230.142869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.522629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.879556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.897222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.078748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7236.124323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7237.132772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.420959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.793563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.087391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7242.918814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.519239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7245.830120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.072393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7248.025809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7249.177201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.418989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7251.722284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.907909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7254.087547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.120566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7256.379419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.080893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.395726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.603090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.375449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7263.772618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7264.943667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7266.306632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7267.813739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.128083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7270.606346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7271.611201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7272.763842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.438389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7276.046664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.846267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7279.564886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.202945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7283.063489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.207570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7286.937827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.732478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.682530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.886898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7294.872945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.372046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7299.278527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7301.083120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.859502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7304.308331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7305.494062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.364691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.602082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7308.981375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.133214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7311.449708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.519767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.724782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7314.932963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.912239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.021062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.297891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.691885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.938590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.276244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.483972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7324.764790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7326.013815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.350585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.362680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.671268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7330.863176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7332.342368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7333.452398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.414593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7335.562957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.823715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.743435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.044903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7340.148304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.293726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.722205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.015364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7345.210969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7346.679900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.024525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.427356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7350.992909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.769085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.136200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7355.523150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7356.785881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7357.801996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7359.022495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.133811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7361.442152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.656982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.100784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7365.375674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.548577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.541225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.829563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.038295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.542870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7372.880128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7374.240551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.375811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7376.661708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.147692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.357501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7380.673442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7381.718698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7382.834759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.961506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.072735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7386.285249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.542199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.821200] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7388.833872] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7388.849202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7389.949428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.989534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.100161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.986888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.111863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.374250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7396.575662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.031802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.160311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.288146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7401.634515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7403.013983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7404.068082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.220855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.202093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7407.410414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.581303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.742844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7411.055693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.243509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7413.487791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.902173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.002506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.159299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7418.278245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.382147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7420.458946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.353077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.264407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7423.012532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.037730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.979853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.186953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7427.483195] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7427.496688] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7427.500453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.627963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.695275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.048502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.006546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.269633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.465607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.048479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.481231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7438.765521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7440.063468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.391035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.732577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7444.449186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7446.065967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.552225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7448.573602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7449.655222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.045420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7452.870283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7454.938027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7456.847984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7458.895145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.557495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7462.366233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.309458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7465.723702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.250713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.566158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.371361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.756511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.033475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7473.617766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.309685] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7475.327186] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7475.378591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.820127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7478.200446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.726379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.201257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7482.670088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.168662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7485.274918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.315908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7487.520566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7488.554900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7489.672118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7490.872666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.475567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.935572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.518703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7496.765159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7498.271632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.686746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.054516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.253227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7503.396029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.787279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.149269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.344288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.347624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7509.313020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.309917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7511.638077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.906350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.695047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.817756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7516.279363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.878741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7519.411461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.057673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.243412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.539469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7524.586317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.896151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.064379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7528.233068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.107119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.275130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7531.403457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.434540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7533.462462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.993967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7536.284349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.562906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7538.852458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.057310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7541.204075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7542.922852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7543.954909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7545.338878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.835205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.961149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.329514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7550.592686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.786312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7553.164538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.355266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7555.735313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.746048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7557.926511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7559.426310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.919721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.056881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7563.364083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7564.954165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.315590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.767662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7569.400474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7571.048332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7571.942179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7573.001536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7574.234984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.238805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.486850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.630130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7578.788954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7579.722379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7580.790132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7581.837101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.901330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7584.143696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7585.619513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7587.146908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7588.609866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.184973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7591.749197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.092688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7594.521276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7595.841422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.817043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7597.834303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.900645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7599.892269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7600.882542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7601.980388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7603.031349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.091914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.961462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7605.794655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7606.708279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7607.810129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7608.917462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7610.094483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.710340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7613.309409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7614.687652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7616.493906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7617.577310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7618.657196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7619.802333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7621.246349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.423599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7624.226686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7626.295644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.057735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7630.116105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.589515] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7632.603839] Lustre: Skipped 392 previous similar messages [ 7632.641863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7632.668232] Lustre: Skipped 392 previous similar messages [ 7632.682905] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7632.701375] Lustre: Skipped 392 previous similar messages [ 7632.705297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.719740] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7632.735835] Lustre: Skipped 392 previous similar messages [ 7635.002476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.869489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7638.373635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.428376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7640.695856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7642.409822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7644.754412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.620431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7648.761388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7650.208484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7651.478198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7653.132948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7655.052895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7656.385685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7657.520186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7658.657655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7659.536729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7660.938430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7662.130099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7663.821270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7665.652913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7667.041372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7668.205612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.390117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7670.579791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7671.649814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7672.595723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.696677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7674.846359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7676.036790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7677.154550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.192587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7679.268677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7680.547555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7681.870782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.070660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7684.223651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7685.441542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7686.541614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7687.519864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7688.771763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.039071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7691.370952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7692.703122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7694.109267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7695.400219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7696.658630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.911754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7699.077280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7700.169978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7701.492711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7702.875812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7704.030971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.321627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.516161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7707.846603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.135178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7710.528144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.035902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.389119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7714.887205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.168090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7717.355358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7718.685189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7720.428232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7722.032502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7723.400410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.402252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7726.696644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7727.943033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7729.056590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.251857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7731.497221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7732.936391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7734.476917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7735.838747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7737.192604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.544018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7739.833466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7741.247425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7742.592994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7743.915210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7745.069951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.522925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7747.745785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7748.819027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7750.004533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7751.353574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7752.585023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7753.946063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7755.062839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7756.203431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7757.529739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7758.641532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.066236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7761.064284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7762.644601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7763.920993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7765.351669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7766.794538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7768.408922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7769.756278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.955022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7772.447623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.295067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7776.033355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.502926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7778.835412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7779.885761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.938887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7782.174931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7783.704430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7784.952717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7786.346145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7787.852256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7789.112145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7790.079236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7791.369779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7792.630118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.667822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7794.859766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7796.088918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7797.398284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7798.970988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7800.493728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7801.824287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7803.049709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.418691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7805.521324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7806.499852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.392215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7810.207936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7812.077441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.796215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7815.203688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7816.478103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7817.906349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7819.365792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7820.861750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7822.036433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.169779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.565153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7825.826410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7827.098138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.373599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7829.581843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7831.159397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7833.032783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7834.160251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.452583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7836.862468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7838.328539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7839.663747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7841.491121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7842.998239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7844.578657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7846.250437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7847.571372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7848.761417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7850.343886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.711478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7852.969136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.355605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7855.631417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7856.773441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7857.842778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7858.842691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7860.173030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7861.756486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7863.178508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7864.544173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7866.090395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7867.670256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7869.261772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7870.599900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7872.118600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.520932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7874.832839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7876.100662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.002650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.969197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7879.139868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7880.304904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.245566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7882.335040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.290165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7884.574051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.566692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7886.490422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7887.571238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7888.749907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7890.197279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.428752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7892.672038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7893.871450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.113830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7896.339535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7897.610337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7898.764823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7900.099548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.806604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7903.131757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.547654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7905.812787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7907.129164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7908.960811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7910.228674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7911.469233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.446785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7913.664180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7915.108233] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7915.129777] LustreError: 172559:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7915.137422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8523.441489] LustreError: 174363:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8523.459916] LustreError: 172559:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 9389.027054] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9389.039196] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9389.054195] Lustre: Skipped 211 previous similar messages [ 9389.067764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9390.583461] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9390.594508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9395.115137] Lustre: server umount lustre-OST0000 complete [ 9401.101325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9401.113799] Lustre: Skipped 3 previous similar messages [ 9406.436917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9406.444627] Lustre: Skipped 1 previous similar message [ 9408.244071] Lustre: server umount lustre-MDT0000 complete [ 9411.553057] LustreError: 172534:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9411.576687] LustreError: 172534:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9414.395967] LustreError: 172518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785477248 with bad export cookie 3304212218294878970 [ 9414.400124] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9414.405793] LustreError: 172518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9415.321321] Lustre: server umount lustre-MDT0001 complete [ 9430.876239] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 9434.943522] Key type lgssc unregistered [ 9435.471410] LNet: 227924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9435.493514] LNetError: 227924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9435.522043] LNet: Removed LNI 192.168.204.105@tcp [ 9436.788370] Key type .llcrypt unregistered [ 9436.792187] Key type ._llcrypt unregistered [ 9468.761486] Key type ._llcrypt registered [ 9468.766598] Key type .llcrypt registered [ 9468.907151] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 9470.749926] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9470.813667] alg: No test for adler32 (adler32-zlib) [ 9472.031885] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 9472.398290] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 9474.199337] Key type lgssc registered [ 9476.327264] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9486.692699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9486.751937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9488.599182] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9494.941083] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9499.118523] LustreError: 229342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9499.151471] LustreError: 229342:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9504.227961] LustreError: 229343:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9507.893360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9508.430897] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9516.061783] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9527.865923] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9536.375204] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9545.816603] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9546.184328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9554.435524] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9559.574703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 9564.918406] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9571.755767] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 9574.606384] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9574.629175] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 9574.639201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9574.654382] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9574.659361] LustreError: 229362:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 9578.185139] Lustre: lustre-OST0000: Not available for connect from 192.168.204.5@tcp (stopping) [ 9578.976259] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9578.991646] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9579.019782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9580.013358] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9580.045563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9580.059616] Lustre: Skipped 1 previous similar message [ 9583.265480] Lustre: lustre-OST0000: Not available for connect from 192.168.204.5@tcp (stopping) [ 9588.387938] Lustre: lustre-OST0000: Not available for connect from 192.168.204.5@tcp (stopping) [ 9588.394704] Lustre: Skipped 2 previous similar messages [ 9591.266583] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9591.786350] Lustre: server umount lustre-OST0000 complete [ 9594.711086] LustreError: 229362:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 9630.074873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9630.585578] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9641.674332] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9645.937651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9645.979892] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9645.987326] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9654.809719] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9657.576864] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 9667.740835] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ee7100ef800.ost_server_uuid 50 [ 9670.569779] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ee7100ef800.ost_server_uuid in IDLE state after 0 sec [ 9676.774498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9676.782972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9676.804140] Lustre: Skipped 1 previous similar message [ 9676.828027] Lustre: Skipped 3 previous similar messages [ 9682.983624] Lustre: server umount lustre-OST0000 complete [ 9692.131153] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9692.147702] Lustre: Skipped 1 previous similar message [ 9695.066401] Lustre: server umount lustre-MDT0000 complete [ 9697.253811] LustreError: 231814:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9697.279657] LustreError: 231814:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9700.275052] LustreError: 229323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785477534 with bad export cookie 7131711713097906095 [ 9700.296149] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9700.901510] Lustre: server umount lustre-MDT0001 complete [ 9717.025602] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 9721.282080] Key type lgssc unregistered [ 9721.737960] LNet: 234560:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9721.754198] LNetError: 234560:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9721.784553] LNet: Removed LNI 192.168.204.105@tcp [ 9722.765157] Key type .llcrypt unregistered [ 9722.768792] Key type ._llcrypt unregistered [ 9764.751175] Key type ._llcrypt registered [ 9764.755372] Key type .llcrypt registered [ 9764.861443] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 9767.189139] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9767.217122] alg: No test for adler32 (adler32-zlib) [ 9768.246410] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [ 9768.638434] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 9770.463148] Key type lgssc registered [ 9771.914116] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9783.515301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9783.545273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9785.157792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9791.177962] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9794.536915] LustreError: 236294:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9794.560811] LustreError: 236294:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9799.653668] LustreError: 236295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9801.997501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9802.625977] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9808.939918] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9813.152676] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9824.449862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9825.823304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9829.954949] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 9838.710317] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9851.422359] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9851.713567] Lustre: lustre-OST0001: new disk, initializing [ 9851.716713] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9851.733572] Lustre: 238616:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9851.825297] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9857.412550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9857.427143] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9857.639574] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9862.443305] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9878.229812] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9887.112899] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9908.712955] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9908.719641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9908.734139] Lustre: Skipped 3 previous similar messages [ 9908.744944] Lustre: Skipped 2 previous similar messages [ 9913.825165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9913.830918] Lustre: Skipped 3 previous similar messages [ 9914.790533] Lustre: server umount lustre-MDT0000 complete [ 9918.971878] LustreError: 237575:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9919.004729] LustreError: 237575:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9919.454529] LustreError: 237805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785477753 with bad export cookie 12836862661742165610 [ 9919.456636] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9919.472811] LustreError: 237805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9920.142463] Lustre: server umount lustre-MDT0001 complete [ 9935.035658] Lustre: server umount lustre-OST0000 complete [ 9939.423163] Lustre: 235558:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785477757/real 1785477757] req@ffff8f707d801f80 x1872208962105728/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785477773 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9939.504219] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9941.525707] Lustre: server umount lustre-OST0001 complete [ 9971.144807] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 9983.733708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9984.492621] LustreError: 241725:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9984.599586] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9989.601287] LustreError: 241726:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9990.283795] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 9999.841826] LustreError: 241725:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9999.860294] LustreError: 241725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10001.608432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10002.087673] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10008.009905] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10012.215669] Lustre: 242868:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10023.767042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10024.229367] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10026.287068] LustreError: 243223:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10026.321609] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [10029.383620] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [10033.751539] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10044.984307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10052.657450] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10063.509398] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10068.760600] Lustre: 244744:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10086.385283] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10086.388767] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10086.406174] Lustre: Skipped 3 previous similar messages [10091.494848] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10091.507904] Lustre: Skipped 6 previous similar messages [10092.864195] Lustre: server umount mylustre-MDT0000 complete [10096.623156] LustreError: 242468:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10096.651545] LustreError: 242468:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [10098.160557] LustreError: 242871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785477931 with bad export cookie 12836862661742168473 [10098.176395] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10098.189815] LustreError: 242871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10099.311447] Lustre: server umount mylustre-MDT0001 complete [10115.550844] Lustre: server umount mylustre-OST0000 complete [10117.216836] Lustre: 235557:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785477935/real 1785477935] req@ffff8f6f60b4a680 x1872208962179584/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785477951 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10117.259885] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10119.967615] Lustre: 235559:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785477937/real 1785477937] req@ffff8f6f5915ea00 x1872208962179840/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785477953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10120.908920] Lustre: server umount mylustre-OST0001 complete [10152.045603] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [10165.038430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10165.443701] LustreError: 247056:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10165.477816] LustreError: 247056:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10165.540202] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10165.551348] Lustre: Skipped 1 previous similar message [10171.806620] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10183.788552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10190.867640] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10196.174100] Lustre: 248199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10207.244167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10209.860181] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [10210.862254] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [10215.872938] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10228.772712] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10234.367053] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [10237.679486] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10248.129906] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10253.577298] Lustre: 250068:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10265.065184] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10265.069723] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [10265.105258] Lustre: Skipped 3 previous similar messages [10265.119237] Lustre: Skipped 3 previous similar messages [10270.312933] Lustre: server umount tfs-MDT0000 complete [10275.188429] LustreError: 247033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785478108 with bad export cookie 12836862661742170426 [10275.194758] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10275.197289] LustreError: 247033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10275.297387] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10275.313871] Lustre: Skipped 1 previous similar message [10275.320468] Lustre: tfs-MDT0001: Not available for connect from 0@lo (stopping) [10275.326920] Lustre: Skipped 1 previous similar message [10275.779883] Lustre: server umount tfs-MDT0001 complete [10281.682959] Lustre: server umount tfs-OST0000 complete [10287.110768] Lustre: server umount tfs-OST0001 complete [10317.155608] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [10331.809333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10333.065797] LustreError: 252320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10333.096722] LustreError: 252320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [10333.250920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10333.253521] Lustre: Skipped 3 previous similar messages [10343.014678] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10358.412806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10367.124566] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10371.049854] Lustre: 253465:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10381.507232] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10384.063882] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [10388.974099] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10398.785573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10404.261510] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [10404.273964] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [10407.138638] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10419.302743] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10426.243769] Lustre: 255342:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10437.131962] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:11:07 (1785478267) [10455.521755] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10455.525908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10455.549482] Lustre: Skipped 2 previous similar messages [10455.552636] Lustre: Skipped 1 previous similar message [10461.831584] Lustre: server umount lustre-MDT0000 complete [10467.262429] LustreError: 254649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785478301 with bad export cookie 12836862661742172344 [10467.265287] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10467.287351] LustreError: 254649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10474.252689] Lustre: server umount lustre-MDT0001 complete [10481.224834] Lustre: server umount lustre-OST0000 complete [10487.085280] Lustre: server umount lustre-OST0001 complete [10500.991185] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [10520.678400] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [10573.921942] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [10573.921942] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10573.921942] [10573.958188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10574.318857] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10574.362564] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10574.495272] Lustre: lustre-MDT0000: new disk, initializing [10574.688996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10574.701571] Lustre: Skipped 3 previous similar messages [10574.760767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10580.540329] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10594.924420] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [10594.924420] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10594.924420] [10594.979485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10595.144282] Lustre: 258548:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10595.214407] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10595.220778] Lustre: Skipped 1 previous similar message [10595.307325] Lustre: lustre-MDT0001: new disk, initializing [10595.460667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10595.471309] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10601.016362] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10615.311049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10615.737110] Lustre: lustre-OST0000: new disk, initializing [10615.743491] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10615.750404] Lustre: 260130:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10617.530589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10617.539313] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10617.745946] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10623.637022] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10636.929605] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10637.170114] Lustre: lustre-OST0001: new disk, initializing [10637.176759] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10637.184213] Lustre: 261003:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10638.258407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10638.270812] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10638.374048] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10645.564497] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10660.276939] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:14:50 (1785478490) [10681.882174] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:15:12 (1785478512) [10704.873387] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10704.893746] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10704.896021] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10704.923973] Lustre: Skipped 4 previous similar messages [10704.945468] Lustre: Skipped 11 previous similar messages [10706.635980] Lustre: server umount lustre-OST0000 complete [10710.012422] LustreError: 260114:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10710.090449] LustreError: 260114:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [10713.060639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10713.069049] LustreError: Skipped 1 previous similar message [10727.391125] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10727.881930] Lustre: server umount lustre-MDT0000 complete [10736.731700] LustreError: 260140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785478570 with bad export cookie 12836862661742174122 [10736.752083] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10736.764975] LustreError: 260140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10743.922075] Lustre: server umount lustre-MDT0001 complete [10760.673946] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [10776.953238] Lustre: server umount lustre-OST0001 complete [10786.464649] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [10809.884412] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [10868.339396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10868.876270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10868.979162] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10869.235073] Lustre: lustre-MDT0000: new disk, initializing [10869.390427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10869.406593] Lustre: Skipped 3 previous similar messages [10869.457095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10875.529641] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10891.134944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10891.468379] Lustre: 265572:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10891.597396] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10891.614044] Lustre: Skipped 1 previous similar message [10891.821831] Lustre: lustre-MDT0001: new disk, initializing [10892.118620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10892.146420] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10901.120542] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10920.697039] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10931.362950] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10939.597292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10939.850863] Lustre: lustre-OST0000: new disk, initializing [10939.854207] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10939.862312] Lustre: 267547:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10941.265724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10941.285339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10941.518552] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10948.645535] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [10965.320235] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10982.372575] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10982.372901] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10982.393133] Lustre: Skipped 4 previous similar messages [10982.416491] LustreError: Skipped 1 previous similar message [10982.421091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10982.437254] Lustre: Skipped 13 previous similar messages [10986.980699] Lustre: server umount lustre-OST0000 complete [10994.656738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10999.478203] Lustre: server umount lustre-MDT0000 complete [11003.788296] LustreError: 265564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785478837 with bad export cookie 12836862661742175592 [11003.794358] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11003.802644] LustreError: 265564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11004.448938] Lustre: server umount lustre-MDT0001 complete [11020.455691] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [11023.746248] Key type lgssc unregistered [11024.191548] LNet: 269324:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11024.201742] LNetError: 269324:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11024.228321] LNet: Removed LNI 192.168.204.105@tcp [11025.435141] Key type .llcrypt unregistered [11025.439255] Key type ._llcrypt unregistered [11065.375251] Key type ._llcrypt registered [11065.376635] Key type .llcrypt registered [11065.561786] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [11067.460813] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11067.528270] alg: No test for adler32 (adler32-zlib) [11068.731427] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [11068.999976] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [11070.671114] Key type lgssc registered [11071.909835] Lustre: Echo OBD driver; http://www.lustre.org/ [11079.593546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11079.626873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11081.205435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11087.413915] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11090.401209] LustreError: 270788:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11090.434424] LustreError: 270788:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11095.525707] LustreError: 270789:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11099.112793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11099.515515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11105.733844] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11117.163922] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11124.392536] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11134.438265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11134.803367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11144.042865] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11151.173418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [11156.007476] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11157.382726] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [11161.582931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11161.584044] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11161.616336] LustreError: Skipped 1 previous similar message [11161.624153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11161.637616] Lustre: Skipped 1 previous similar message [11166.635225] Lustre: server umount lustre-OST0000 complete [11176.417475] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11176.430099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11176.450108] Lustre: Skipped 1 previous similar message [11176.453233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11182.052267] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11182.071235] Lustre: Skipped 2 previous similar messages [11187.169055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11187.179379] Lustre: Skipped 1 previous similar message [11188.191153] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11188.694166] Lustre: server umount lustre-MDT0000 complete [11192.292763] LustreError: 270785:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11192.320883] LustreError: 270785:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11194.891205] LustreError: 270769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785479028 with bad export cookie 10757248244781048243 [11194.917585] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11194.930915] LustreError: 270769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11196.227131] Lustre: server umount lustre-MDT0001 complete [11224.414683] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [11232.189951] Key type lgssc unregistered [11232.707528] LNet: 274403:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11232.731022] LNetError: 274403:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11232.750939] LNet: Removed LNI 192.168.204.105@tcp [11233.915323] Key type .llcrypt unregistered [11233.918398] Key type ._llcrypt unregistered [11260.834319] Key type ._llcrypt registered [11260.837970] Key type .llcrypt registered [11261.033903] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [11263.319597] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11263.368481] alg: No test for adler32 (adler32-zlib) [11264.723976] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [11265.269348] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [11267.071169] Key type lgssc registered [11268.708771] Lustre: Echo OBD driver; http://www.lustre.org/ [11281.337291] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11281.380071] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11297.184177] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [11302.303407] LustreError: 275599:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.105@tcp: failed processing log, type 4: rc = -110 [11328.991610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11337.150758] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11347.326949] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11350.270615] Lustre: server umount lustre-OST0000 complete [11375.361970] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [11380.096547] Key type lgssc unregistered [11380.550146] LNet: 277233:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11380.559207] LNetError: 277233:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11380.575765] LNet: Removed LNI 192.168.204.105@tcp [11381.848531] Key type .llcrypt unregistered [11381.851597] Key type ._llcrypt unregistered [11449.400344] Key type ._llcrypt registered [11449.406857] Key type .llcrypt registered [11449.620814] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [11452.429021] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11452.503663] alg: No test for adler32 (adler32-zlib) [11453.968443] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [11454.360546] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [11456.120076] Key type lgssc registered [11457.707969] Lustre: Echo OBD driver; http://www.lustre.org/ [11479.128324] loop0: detected capacity change from 0 to 204800000 [11479.379784] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11479.395434] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11479.887901] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11481.203904] loop0: detected capacity change from 0 to 204800000 [11481.497210] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11481.511920] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11482.241993] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11483.825128] loop0: detected capacity change from 0 to 204800000 [11484.161264] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11484.180494] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11484.456920] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11485.998714] loop0: detected capacity change from 0 to 204800000 [11486.166407] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11486.185635] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11486.284163] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11487.710194] loop0: detected capacity change from 0 to 204800000 [11487.769491] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11493.529520] loop0: detected capacity change from 0 to 204800000 [11493.589756] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11500.823675] loop0: detected capacity change from 0 to 204800000 [11500.865793] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11506.139677] loop0: detected capacity change from 0 to 204800000 [11506.210914] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11511.312060] loop0: detected capacity change from 0 to 204800000 [11511.423901] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11511.569749] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11511.674905] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11511.685826] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [11511.709843] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [11511.721844] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [11511.733282] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [11511.752664] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [11511.782804] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [11511.799531] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [11511.819178] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [11511.830709] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [11511.851982] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [11511.864318] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [11511.881650] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [11511.894586] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [11519.785530] Lustre: server umount MGS complete [11521.219603] loop0: detected capacity change from 0 to 204800000 [11521.412519] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11522.471033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11522.588727] Lustre: lustre-MDT0000: Aborting client recovery [11523.869513] loop1: detected capacity change from 0 to 204800000 [11524.122957] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11524.311736] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11524.354536] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [11524.413387] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [11524.455645] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [11524.531433] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [11524.552378] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [11524.586382] Lustre: 281089:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [11524.652549] Lustre: Found index 1 for lustre-MDT0001, updating log [11525.148152] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11525.239801] Lustre: lustre-MDT0001: Aborting client recovery [11527.158129] loop2: detected capacity change from 0 to 204800000 [11527.492532] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11527.593954] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11527.640438] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11527.675436] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11527.704249] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11527.756740] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [11527.787750] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [11527.832780] Lustre: 281169:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [11528.048669] Lustre: Found index 0 for lustre-OST0000, updating log [11528.134499] Lustre: 281187:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11528.284995] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11528.327289] Lustre: lustre-OST0000: Aborting recovery [11529.615842] loop3: detected capacity change from 0 to 204800000 [11529.815289] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11529.981613] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11530.014480] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [11530.037311] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11530.054897] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [11530.072978] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11530.090187] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11530.110170] Lustre: 281252:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [11530.152510] Lustre: Found index 1 for lustre-OST0001, updating log [11530.213616] Lustre: 281260:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11530.314325] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11530.342110] LustreError: 281032:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [11530.342944] Lustre: lustre-OST0001: Aborting recovery [11532.345782] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11532.367512] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11532.514201] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [11532.563462] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [11534.445981] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [11534.450883] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [11541.605995] LustreError: 280999:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2074us: rc = -2 [11542.134577] LustreError: 280996:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1053us: rc = -2 [11542.168276] LustreError: 280996:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [11543.162971] LustreError: 281494:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 458us: rc = -2 [11543.190496] LustreError: 281494:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [11555.812316] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11555.820535] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11555.839696] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11560.419370] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11560.421468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11560.445665] Lustre: Skipped 2 previous similar messages [11560.461284] Lustre: Skipped 2 previous similar messages [11562.351935] Lustre: server umount lustre-MDT0000 complete [11563.594134] LustreError: 280984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785479397 with bad export cookie 8109125544011802396 [11563.620393] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11565.539350] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11565.542126] LustreError: 280996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11565.542263] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11565.542268] Lustre: Skipped 1 previous similar message [11565.565383] Lustre: Skipped 1 previous similar message [11565.583683] LustreError: 280996:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11568.545437] LustreError: 280998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11568.548896] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11568.583104] Lustre: Skipped 2 previous similar messages [11570.588490] Lustre: server umount lustre-MDT0001 complete [11578.311638] Lustre: server umount lustre-OST0000 complete [11586.537509] Lustre: server umount lustre-OST0001 complete [11598.073222] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:30:29 (1785479429) [11632.666542] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [11645.842186] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [11703.246125] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11703.896822] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11703.983614] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11704.102261] Lustre: lustre-MDT0000: new disk, initializing [11704.201587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11704.225161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11710.529780] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11723.251181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11723.394336] Lustre: 285332:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11723.477419] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11723.483621] Lustre: Skipped 1 previous similar message [11723.595482] Lustre: lustre-MDT0001: new disk, initializing [11723.714924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11723.776024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11723.810771] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11731.269491] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11747.319822] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11759.400865] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11771.920752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11772.649100] Lustre: lustre-OST0000: new disk, initializing [11772.657934] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11772.662892] Lustre: 287302:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11772.864432] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11780.706882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11780.721449] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11780.941965] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11783.342199] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11796.432891] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11802.735851] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11814.665646] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11828.688798] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11841.175731] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [11898.859386] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11898.870585] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11898.881566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11898.899117] Lustre: Skipped 1 previous similar message [11903.083544] Lustre: server umount lustre-OST0000 complete [11913.188806] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11913.202784] LustreError: Skipped 1 previous similar message [11913.205483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11913.213635] Lustre: Skipped 1 previous similar message [11913.218851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11916.176615] Lustre: server umount lustre-MDT0000 complete [11918.308246] LustreError: 285337:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11918.335210] LustreError: 285337:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11923.118044] LustreError: 288931:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785479756 with bad export cookie 8109125544011804762 [11923.122130] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11923.128240] LustreError: 288931:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11923.929661] Lustre: server umount lustre-MDT0001 complete [11940.392134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11946.175311] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11953.002137] Lustre: server umount MGS complete [11969.381149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11969.897308] LustreError: 291299:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11969.920833] LustreError: 291312:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11969.991589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11975.144432] LustreError: 291300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11975.720849] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [11984.358373] LustreError: 291299:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11984.381031] LustreError: 291299:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11988.288624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11995.873804] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12004.674119] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12012.717120] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12022.514883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12033.074217] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12044.394132] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12085.728987] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12085.730975] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12085.735341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12085.735349] Lustre: Skipped 2 previous similar messages [12085.748320] LustreError: Skipped 1 previous similar message [12085.797214] Lustre: Skipped 2 previous similar messages [12095.967454] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12096.562676] Lustre: server umount lustre-OST0000 complete [12106.218284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12106.254024] Lustre: Skipped 1 previous similar message [12108.496329] Lustre: server umount lustre-MDT0000 complete [12111.328933] LustreError: 291295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12111.426985] LustreError: 291295:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12114.849458] LustreError: 291280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785479948 with bad export cookie 8109125544011806715 [12114.856267] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12114.877692] LustreError: 291280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12115.688136] Lustre: server umount lustre-MDT0001 complete [12136.527172] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [12141.727908] Key type lgssc unregistered [12142.285123] LNet: 295273:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12142.306384] LNetError: 295273:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12142.349164] LNet: Removed LNI 192.168.204.105@tcp [12144.003648] Key type .llcrypt unregistered [12144.006965] Key type ._llcrypt unregistered [12202.693645] Key type ._llcrypt registered [12202.696521] Key type .llcrypt registered [12202.846238] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [12224.430816] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [12226.286657] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12226.363041] alg: No test for adler32 (adler32-zlib) [12228.145957] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [12228.682510] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [12230.455174] Key type lgssc registered [12232.710857] Lustre: Echo OBD driver; http://www.lustre.org/ [12283.323438] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12283.386105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12284.967955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12285.137384] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12285.354995] Lustre: lustre-MDT0000: new disk, initializing [12285.537759] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12285.614474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12292.877301] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12308.845205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12309.007055] Lustre: 299253:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12309.086651] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12309.090343] Lustre: Skipped 1 previous similar message [12309.184801] Lustre: lustre-MDT0001: new disk, initializing [12309.344334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12309.425177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12309.447553] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12314.833387] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12328.015386] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12334.826513] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12343.530914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12343.908464] Lustre: lustre-OST0000: new disk, initializing [12343.915280] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12343.926322] Lustre: 301221:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12344.004359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12351.645444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12351.663546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12351.744509] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12352.155195] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12367.916631] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12378.490522] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12390.599978] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12395.934954] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12401.227588] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [12449.255304] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12449.266127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12449.271302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12449.274871] LustreError: Skipped 1 previous similar message [12449.322114] Lustre: Skipped 1 previous similar message [12453.781129] Lustre: server umount lustre-OST0000 complete [12459.268172] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12459.300804] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12459.319372] Lustre: Skipped 2 previous similar messages [12464.608769] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12464.615106] Lustre: Skipped 1 previous similar message [12466.103304] Lustre: server umount lustre-MDT0000 complete [12471.102145] LustreError: 299246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785480304 with bad export cookie 7432656716947180837 [12471.106909] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12471.112351] LustreError: 299246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12471.646947] Lustre: server umount lustre-MDT0001 complete [12482.255113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12487.488442] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12493.147915] Lustre: server umount MGS complete [12506.910172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12507.466426] LustreError: 304995:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12507.491645] LustreError: 304995:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12507.523498] LustreError: 305008:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12507.620352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12512.737046] LustreError: 304996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12513.727894] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12517.864447] LustreError: 304995:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12522.982647] LustreError: 304996:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12523.995849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12524.468132] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12530.144362] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12541.194957] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12551.129810] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12563.536207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12564.048597] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12573.725156] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12583.651828] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12621.796291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12621.823617] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12621.859675] Lustre: Skipped 1 previous similar message [12621.878616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12623.841739] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12625.991573] Lustre: server umount lustre-OST0000 complete [12632.034797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12632.049229] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12632.069930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12632.079714] Lustre: Skipped 2 previous similar messages [12637.105000] Lustre: server umount lustre-MDT0000 complete [12639.204279] LustreError: 307445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12639.236303] LustreError: 307445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12642.823927] LustreError: 304977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785480476 with bad export cookie 7432656716947182692 [12642.826686] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12642.837897] LustreError: 304977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12643.471308] Lustre: server umount lustre-MDT0001 complete [12658.991869] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [12663.739170] Key type lgssc unregistered [12664.102599] LNet: 308944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12664.112446] LNetError: 308944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12664.129748] LNet: Removed LNI 192.168.204.105@tcp [12665.034384] Key type .llcrypt unregistered [12665.038682] Key type ._llcrypt unregistered [12701.677556] Key type ._llcrypt registered [12701.679567] Key type .llcrypt registered [12701.867640] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [12720.078248] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [12721.796603] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12721.894695] alg: No test for adler32 (adler32-zlib) [12723.023727] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [12723.264730] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [12725.015171] Key type lgssc registered [12726.419088] Lustre: Echo OBD driver; http://www.lustre.org/ [12757.274640] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [12786.005122] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12786.044299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12787.621146] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12787.680811] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12788.028125] Lustre: lustre-MDT0000: new disk, initializing [12788.261935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12788.342747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12796.907315] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [12812.631040] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12813.077750] Lustre: lustre-OST0000: new disk, initializing [12813.088750] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12813.098231] Lustre: Skipped 1 previous similar message [12813.107961] Lustre: 313008:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12813.275027] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12822.240173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [12822.255160] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [12822.323465] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [12824.165460] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14103.016961] Lustre: server umount lustre-MDT0000 complete [14108.047274] LustreError: 312192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785481941 with bad export cookie 17458252370266011558 [14108.069739] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14117.799069] Lustre: server umount lustre-OST0000 complete [14154.399419] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [14176.885266] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [14244.913270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14245.459502] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14245.513780] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14245.706267] Lustre: lustre-MDT0000: new disk, initializing [14245.915951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14245.951195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14254.412853] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14268.487483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14268.722143] Lustre: 317445:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14268.812455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14268.820976] Lustre: Skipped 1 previous similar message [14268.967466] Lustre: lustre-MDT0001: new disk, initializing [14269.113500] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14269.184326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14269.203130] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14275.836091] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14289.202112] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14297.914393] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14309.046637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14309.580352] Lustre: lustre-OST0000: new disk, initializing [14309.597539] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14309.628938] Lustre: 319415:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14309.757888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14318.169887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14318.183585] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14318.264941] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14318.340908] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14332.204843] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14339.267255] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14339.655821] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14346.451302] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [14346.924243] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14354.405233] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14354.407262] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14354.430043] LustreError: Skipped 1 previous similar message [14354.431052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14354.469090] Lustre: Skipped 1 previous similar message [14356.181196] Lustre: server umount lustre-OST0000 complete [14364.642997] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14364.644922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14364.673485] Lustre: Skipped 1 previous similar message [14364.704823] Lustre: Skipped 2 previous similar messages [14369.761241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14369.782632] Lustre: Skipped 1 previous similar message [14370.503508] Lustre: server umount lustre-MDT0000 complete [14378.424536] LustreError: 317438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785482212 with bad export cookie 17458252370268952405 [14378.429358] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14378.452371] LustreError: 317438:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14379.276442] Lustre: server umount lustre-MDT0001 complete [14384.080209] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [14386.981198] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:16:57 (1785482217) [14397.562264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14398.310552] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14398.349077] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [14398.532126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14403.553589] LustreError: 321732:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14408.695398] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14410.568233] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14413.797302] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14418.912185] LustreError: 321732:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14429.160718] LustreError: 321732:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14429.205786] LustreError: 321732:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14432.264856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14433.203078] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14440.312404] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14449.856204] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14457.409943] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14468.902016] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14469.583648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14479.361502] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14491.971248] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14501.226459] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14501.485679] Lustre: lustre-OST0001: new disk, initializing [14501.491942] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [14501.500554] Lustre: 324576:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14501.636929] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14509.958638] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14510.727262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [14510.738230] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [14510.827795] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14511.909970] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14511.930062] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [14512.937859] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14515.106343] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14515.128263] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [14519.280971] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14519.308203] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [14523.400108] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14523.671425] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14527.599063] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14527.618582] LustreError: 324955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [14531.785163] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ee710352000.ost_server_uuid 50 [14533.948406] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ee710352000.ost_server_uuid in FULL state after 0 sec [14541.830579] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ee710352000.ost_server_uuid 50 [14543.570840] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8ee710352000.ost_server_uuid in IDLE state after 0 sec [14544.035142] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14544.052054] LustreError: 324963:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 30 previous similar messages [14546.258621] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [14561.765209] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14561.774605] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14561.795297] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [14561.801083] Lustre: Skipped 1 previous similar message [14561.825107] Lustre: Skipped 1 previous similar message [14566.329244] Lustre: server umount lustre-OST0001 complete [14567.423299] LustreError: 324956:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14567.462445] LustreError: 324956:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14576.101407] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14576.107652] LustreError: Skipped 1 previous similar message [14576.120690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14576.156613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14576.168101] Lustre: Skipped 3 previous similar messages [14582.186954] Lustre: server umount lustre-OST0000 complete [14587.873958] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14587.891301] Lustre: Skipped 1 previous similar message [14587.895950] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14587.901518] Lustre: Skipped 3 previous similar messages [14602.207165] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14602.864496] Lustre: server umount lustre-MDT0000 complete [14603.247286] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14603.279197] LustreError: 321731:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [14609.216529] LustreError: 321714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785482443 with bad export cookie 17458252370268953287 [14609.218341] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14609.229887] LustreError: 321714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14609.901632] Lustre: server umount lustre-MDT0001 complete [14628.798785] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [14633.200671] Key type lgssc unregistered [14633.563442] LNet: 327293:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14633.576276] LNetError: 327293:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14633.599664] LNet: Removed LNI 192.168.204.105@tcp [14634.670212] Key type .llcrypt unregistered [14634.674546] Key type ._llcrypt unregistered [14678.475585] Key type ._llcrypt registered [14678.477569] Key type .llcrypt registered [14678.573813] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [14679.962969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [14680.121858] alg: No test for adler32 (adler32-zlib) [14681.215572] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [14681.447120] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [14683.135521] Key type lgssc registered [14684.416872] Lustre: Echo OBD driver; http://www.lustre.org/ [14693.818808] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [14693.832457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14695.383348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14699.891452] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14705.645890] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14705.674072] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [14709.853525] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14710.760505] LustreError: 328720:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14715.879756] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14719.199808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14719.982037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14727.718545] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14743.620495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14744.279615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14745.322063] LustreError: 330400:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14750.698877] LustreError: 330454:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14752.090185] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14760.943253] LustreError: 330402:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14760.961069] LustreError: 330402:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14761.810739] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14766.081725] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [14770.822960] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14771.056875] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14776.332825] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [14778.741672] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [14788.597958] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14798.277525] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [14798.424695] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14929.896446] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14929.897817] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14929.902461] LustreError: Skipped 1 previous similar message [14929.903534] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [14929.923690] Lustre: Skipped 1 previous similar message [14932.537340] Lustre: server umount lustre-OST0001 complete [14935.015557] LustreError: 330454:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14935.046941] LustreError: 330454:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14945.256354] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14945.266117] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14945.281703] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14945.283702] Lustre: Skipped 1 previous similar message [14945.305974] Lustre: Skipped 2 previous similar messages [14948.618440] Lustre: server umount lustre-OST0000 complete [14955.492727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14955.493063] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14955.530475] LustreError: Skipped 1 previous similar message [14955.620540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14955.644761] Lustre: Skipped 1 previous similar message [14960.608252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14960.622683] Lustre: Skipped 1 previous similar message [14965.730040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14965.744732] Lustre: Skipped 1 previous similar message [14969.311148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14969.799618] Lustre: server umount lustre-MDT0000 complete [14970.863904] LustreError: 328716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14970.883327] LustreError: 328716:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [14976.918992] LustreError: 328699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785482810 with bad export cookie 8711874060295582541 [14976.931189] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14976.944914] LustreError: 328699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14978.195132] Lustre: server umount lustre-MDT0001 complete [14995.333247] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [14999.610388] Key type lgssc unregistered [15000.098567] LNet: 333674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15000.107033] LNetError: 333674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15000.126638] LNet: Removed LNI 192.168.204.105@tcp [15001.738658] Key type .llcrypt unregistered [15001.739945] Key type ._llcrypt unregistered [15069.205621] Key type ._llcrypt registered [15069.207181] Key type .llcrypt registered [15069.339279] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [15070.809283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [15070.856709] alg: No test for adler32 (adler32-zlib) [15072.457465] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [15072.730078] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [15074.511186] Key type lgssc registered [15075.977453] Lustre: Echo OBD driver; http://www.lustre.org/ [15089.973488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [15089.991865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15091.663527] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15099.153796] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15101.923142] LustreError: 335969:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15101.954165] LustreError: 335969:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15107.047976] LustreError: 335970:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15110.401512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15110.998904] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15116.101137] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15119.629133] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [15129.096686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15129.676661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15135.853594] LustreError: 337459:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15137.649216] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15137.891858] LustreError: 337459:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15137.925670] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [15137.926344] LustreError: 337459:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [15143.397481] LustreError: 337461:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15143.433613] LustreError: 337461:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [15150.875479] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15151.099708] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15156.233023] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [15162.282848] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15174.800992] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15184.161739] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [15203.296956] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15203.309657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15203.331853] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15207.395012] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15207.400966] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15207.444850] Lustre: Skipped 1 previous similar message [15209.463557] Lustre: server umount lustre-MDT0000 complete [15212.516849] LustreError: 335969:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15212.550438] LustreError: 335969:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15215.425429] LustreError: 335951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785483049 with bad export cookie 4167352256785109087 [15215.442318] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15215.460423] LustreError: 335951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [15216.302972] Lustre: server umount lustre-MDT0001 complete [15233.889019] Lustre: 335231:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483051/real 1785483051] req@ffff8f6f485ff480 x1872214523718784/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785483067 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15233.936735] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15233.959989] Lustre: Skipped 1 previous similar message [15234.459876] Lustre: server umount lustre-OST0000 complete [15236.831389] Lustre: 335233:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483054/real 1785483054] req@ffff8f6f485fd180 x1872214523719040/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785483070 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15239.088751] Lustre: 335231:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483056/real 1785483056] req@ffff8f70508a5500 x1872214523719296/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785483072 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15241.251341] Lustre: server umount lustre-OST0001 complete [15266.717228] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [15282.232579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15282.982865] LustreError: 341241:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15283.056221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15289.465723] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15304.728334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15305.494150] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15312.719323] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15319.138765] Lustre: 342383:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [15330.289377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15330.951572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15331.964223] LustreError: 342737:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15331.989362] LustreError: 342737:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15343.118880] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15352.843247] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [15355.854497] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15361.557906] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [15365.646043] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15379.328852] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15388.411232] Lustre: 344258:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [15401.085493] Lustre: server umount lustre-MDT0000 complete [15402.501486] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15402.510511] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15402.544617] LustreError: 341237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15402.561093] Lustre: Skipped 2 previous similar messages [15402.634508] LustreError: 341237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [15407.260742] LustreError: 341224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785483241 with bad export cookie 4167352256785110788 [15407.283173] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15407.588205] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15407.602442] Lustre: Skipped 1 previous similar message [15407.614137] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15407.619651] Lustre: Skipped 3 previous similar messages [15408.017691] Lustre: server umount lustre-MDT0001 complete [15414.114851] Lustre: server umount lustre-OST0000 complete [15421.405095] Lustre: server umount lustre-OST0001 complete [15427.569530] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [15430.429938] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [15433.458628] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:34:23 (1785483263) [15463.988630] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [15476.463279] loop0: detected capacity change from 0 to 1099511627776 [15477.753651] SGI XFS with ACLs, security attributes, quota, no debug enabled [15477.770061] *** VALIDATE xfs *** [15477.803426] XFS (loop0): Mounting V5 Filesystem [15477.842343] XFS (loop0): Ending clean mount [15480.109449] loop1: detected capacity change from 0 to 18691697672192 [15654.159177] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [15674.031375] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [15723.498219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15723.995333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [15724.066306] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [15724.251373] Lustre: lustre-MDT0000: new disk, initializing [15724.382077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15724.407624] Lustre: Skipped 1 previous similar message [15724.463633] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [15732.754737] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15755.115525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15755.370286] Lustre: 349433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [15755.447538] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [15755.452985] Lustre: Skipped 1 previous similar message [15755.667985] Lustre: lustre-MDT0001: new disk, initializing [15756.098167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [15756.126344] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [15765.944976] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15780.367067] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15787.506329] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15796.548338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15797.034309] Lustre: lustre-OST0000: new disk, initializing [15797.044367] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [15797.053601] Lustre: 351406:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [15797.144260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15797.152779] Lustre: Skipped 1 previous similar message [15798.935656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [15798.953806] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [15799.121808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [15806.042095] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15820.086486] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15828.291030] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [15828.694968] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [15834.446748] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [15834.734833] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [15840.250582] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [15840.251247] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15840.263789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15840.263797] Lustre: Skipped 1 previous similar message [15840.315355] Lustre: Skipped 2 previous similar messages [15843.578658] Lustre: server umount lustre-OST0000 complete [15850.467255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15850.478023] Lustre: Skipped 2 previous similar messages [15854.892947] Lustre: server umount lustre-MDT0000 complete [15855.593244] LustreError: 349440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15855.610456] LustreError: 349440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [15859.796060] LustreError: 351405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785483693 with bad export cookie 4167352256785112342 [15859.801816] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15859.821232] LustreError: 351405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [15860.655372] Lustre: server umount lustre-MDT0001 complete [15876.207415] XFS (loop0): Unmounting Filesystem [15884.334939] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:41:53 (1785483713) [15895.995689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15901.947190] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15912.739639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15919.125567] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15930.367455] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15938.704089] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15948.451097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15949.110430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15949.124144] Lustre: Skipped 2 previous similar messages [15958.273545] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [15970.500447] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15987.176140] Lustre: server umount lustre-OST0000 complete [15997.410060] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15997.418620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15997.436428] Lustre: Skipped 3 previous similar messages [15997.464483] Lustre: Skipped 1 previous similar message [16002.464289] Lustre: 335232:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483820/real 1785483820] req@ffff8f7046cffb80 x1872214523867648/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1785483836 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [16003.487324] Lustre: 335230:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483821/real 1785483821] req@ffff8f7067e4a680 x1872214523867904/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1785483837 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 [16005.749742] Lustre: server umount lustre-MDT0000 complete [16008.672588] Lustre: 335230:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785483826/real 1785483826] req@ffff8f6f5118ad80 x1872214523868928/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1785483842 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [16011.566303] LustreError: 353951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785483845 with bad export cookie 4167352256785113224 [16011.592981] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16011.974618] Lustre: server umount lustre-MDT0001 complete [16030.700300] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [16035.935337] Key type lgssc unregistered [16036.854197] LNet: 357628:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16036.910133] LNetError: 357628:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [16036.965879] LNet: Removed LNI 192.168.204.105@tcp [16039.720135] Key type .llcrypt unregistered [16039.727814] Key type ._llcrypt unregistered [16089.228147] Key type ._llcrypt registered [16089.234610] Key type .llcrypt registered [16089.429375] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [16091.386698] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [16091.689331] alg: No test for adler32 (adler32-zlib) [16092.968701] Lustre: Lustre: Build Version: 2.17.56_1_gdebb130 [16093.361822] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [16095.095548] Key type lgssc registered [16097.068128] Lustre: Echo OBD driver; http://www.lustre.org/ [16107.505289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [16107.540759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16109.587206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16116.586050] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16119.778037] LustreError: 359200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16119.795337] LustreError: 359200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16124.903505] LustreError: 359201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16128.307454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16129.069452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16135.286591] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16146.302208] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16155.667926] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16166.446050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16166.887720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [16174.985800] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16184.687777] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16195.554310] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16195.563654] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16195.579509] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16195.589036] Lustre: Skipped 1 previous similar message [16195.617946] Lustre: Skipped 1 previous similar message [16198.325206] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.5@tcp (stopping) [16200.311875] Lustre: server umount lustre-MDT0001 complete [16200.692412] LustreError: 361043:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16203.420330] LustreError: 359196:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.5@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16203.448649] LustreError: 359196:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [16208.560149] LustreError: 359197:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.5@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16208.583477] LustreError: 359197:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16215.401386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16215.557665] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16215.562613] Lustre: Found index 1 for lustre-MDT0001, updating log [16215.609928] Lustre: 359190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16216.037184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16221.190110] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16222.226898] LustreError: 362598:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16222.239458] LustreError: 362598:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16222.271043] LustreError: 359192:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16222.386427] LustreError: 362603:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.105@tcp: cfg command failed: rc = -17 [16222.403523] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [16222.403523] [16223.729577] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16241.639425] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16241.647412] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16241.680345] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16241.696021] Lustre: Skipped 1 previous similar message [16244.421342] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.5@tcp (stopping) [16246.770784] LustreError: 359201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16246.800191] LustreError: 359201:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [16247.495819] Lustre: server umount lustre-MDT0001 complete [16262.845878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16262.954337] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16262.960631] Lustre: Found index 1 for lustre-MDT0001, updating log [16263.047090] Lustre: 359190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16263.472634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16268.795912] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16270.054124] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16270.834283] LustreError: 363757:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16270.844272] LustreError: 363757:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16270.852874] LustreError: 359192:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16270.857760] LustreError: 359192:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16270.915759] LustreError: 363765:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [16270.921991] LustreError: 363765:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [16270.924464] LustreError: 363765:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.105@tcp: cfg command failed: rc = -17 [16270.934611] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.105@tcp [16270.934611] [16577.215283] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.5@tcp (stopping) [16581.091281] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16581.092058] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16581.159713] Lustre: Skipped 2 previous similar messages [16586.208492] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16586.221167] Lustre: Skipped 4 previous similar messages [16591.327165] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [16591.828275] Lustre: server umount lustre-MDT0001 complete [16592.540479] LustreError: 359195:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.5@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16592.569656] LustreError: 359195:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [16607.126538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16607.249270] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16607.255394] Lustre: Found index 1 for lustre-MDT0001, updating log [16607.334896] Lustre: 359190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16607.889584] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [16613.364065] LustreError: 370997:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16613.383214] LustreError: 370997:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16613.399028] LustreError: 359192:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16613.401891] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16613.409230] LustreError: 359192:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16613.511215] LustreError: 371002:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [16613.544896] LustreError: 371002:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.105@tcp: cfg command failed: rc = -114 [16613.570915] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [16613.570915] [16614.481770] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [16940.513401] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16940.531840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16940.564672] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16940.575375] Lustre: Skipped 3 previous similar messages [16946.146319] LustreError: 359201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16946.158705] Lustre: server umount lustre-MDT0000 complete [16946.171254] LustreError: 359201:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [16952.598575] LustreError: 359181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785484786 with bad export cookie 17671786316137327712 [16952.628246] LustreError: 359181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [16952.631042] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16956.385112] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16956.410182] Lustre: Skipped 2 previous similar messages [16959.911378] Lustre: server umount lustre-MDT0001 complete [16970.124779] Lustre: server umount lustre-OST0000 complete [16985.014433] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [17001.103551] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [17054.317633] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:01:24 (1785484884) [17064.937213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17065.389609] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [17065.444080] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [17065.590718] Lustre: lustre-MDT0000: new disk, initializing [17065.717097] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [17065.777415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [17071.773811] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [17086.250635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17086.353102] Lustre: 380376:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [17086.395859] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [17086.398413] Lustre: Skipped 1 previous similar message [17086.451976] Lustre: lustre-MDT0001: new disk, initializing [17086.591179] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [17086.618231] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [17086.632881] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [17092.962619] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [17106.907772] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [17114.341325] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [17124.049928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [17124.448593] Lustre: lustre-OST0000: new disk, initializing [17124.457970] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [17124.467775] Lustre: 382349:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [17124.585993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [17132.233969] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [17132.681508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [17132.690810] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [17132.835442] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [17143.897648] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [17158.114105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [17158.122079] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17158.154746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [17158.161225] Lustre: Skipped 5 previous similar messages [17160.720112] Lustre: server umount lustre-MDT0000 complete [17163.747694] LustreError: 380385:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [17163.770551] LustreError: 380385:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [17166.051164] LustreError: 380371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785484999 with bad export cookie 17671786316137329672 [17166.059778] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [17166.067153] LustreError: 380371:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [17166.819249] Lustre: server umount lustre-MDT0001 complete [17183.169600] Lustre: server umount lustre-OST0000 complete [17226.823984] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [17242.070529] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [17294.257967] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 16981 sec ========= 04:05:24 (1785485124) [17296.975526] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:05:27 (1785485127) === [17303.485172] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:05:33 (1785485133) === [17370.254816] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [17373.775064] Key type lgssc unregistered [17374.219058] LNet: 388747:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17374.225770] LNetError: 388747:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [17374.250774] LNet: Removed LNI 192.168.204.105@tcp [17375.846527] Key type .llcrypt unregistered [17375.850864] Key type ._llcrypt unregistered